# Automorphic All-Pass Recurrent Layer

- ID: 909
- Canonical URL: https://synthcore.org/idea/909/automorphic-all-pass-recurrent-layer
- API JSON: https://synthcore.org/api/idea/909.json
- API Markdown: https://synthcore.org/api/idea/909.md
- Verification status: unverified
- Source: [arXiv:2607.14372](https://arxiv.org/abs/2607.14372)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: rnn, ssm, world-model
- Math tags: linear-algebra, functional-analysis, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Parameterize a recurrent or state-space layer by a matrix-valued Blaschke lift instead of an unconstrained transition matrix. The resulting causal filter is contractive for inputs inside the unit disk and energy-preserving on the unit circle, while its value at z=0 is a freely learned strict contraction.

## Mathematical statement

For a strict contraction T with singular-value factorization T=U diag(a_1,...,a_n)V^*, U and V unitary and |a_j|<1, define the scalar disk automorphism b_a(z)=(z+a)/(1+\overline{a}z). The Nelson lift is \widehat{T}(z)=U diag(b_{a_1}(z),...,b_{a_n}(z))V^*. Because b_a(0)=a, \widehat{T}(0)=T. For |z|<1, |b_a(z)|<1, so the lift is contractive; for |z|=1, |b_a(z)|=1, so the diagonal factor and therefore \widehat{T}(z) are unitary. The causal expansion b_a(z)=a+(1-|a|^2)\sum_{k=1}^{\infty}(-\overline{a})^{k-1}z^k gives an implementable first-order stable recurrence, with memory decay controlled by |a|.

## Key formulas

- $$T=U\begin{bmatrix}s_1&&\\&\ddots&\\&&s_n\end{bmatrix}V^*$$
- $$\widetilde{T}(z_1,\ldots,z_n)=U\begin{bmatrix}z_1&&\\&\ddots&\\&&z_n\end{bmatrix}V^*$$
- $$\widehat{T}(z)=U\operatorname{diag}\left(\frac{z+a_1}{1+\overline{a_1}z},\ldots,\frac{z+a_n}{1+\overline{a_n}z}\right)V^*,\qquad \widehat{T}(0)=T$$
- $$b_a(z)=a+(1-|a|^2)\sum_{k=1}^{\infty}(-\overline{a})^{k-1}z^k,\qquad |b_a(e^{i\theta})|=1$$

## Implementation notes

Integrate the construction as the temporal transition in a recurrent block or diagonal state-space layer acting on feature vectors x_t. Represent complex values with paired real channels if the model is real-valued. Trainable parameters are unitary mixing matrices U and V and scalar parameters alpha_j, phi_j, with a_j=tanh(alpha_j) exp(i phi_j), guaranteeing |a_j|<1. For each timestep, compute v_t=V^*x_t. Maintain one memory value q_{j,t} per channel and update q_{j,t}=-conj(a_j)q_{j,t-1}+v_{j,t-1}. Then compute w_{j,t}=a_jv_{j,t}+(1-|a_j|^2)q_{j,t} and return y_t=Uw_t. This recurrence is the causal realization of the displayed power-series formula; no infinite sum is evaluated. Parameterize U and V with Householder products or a Cayley transform, and optionally add a gated residual y'_t=g y_t+(1-g)x_t with g in [0,1]. The mathematical guarantees are computed exactly from the parameterization, while memory timescales are learned through |a_j|. First test a 256-state version on adding-problem sequences, sequential MNIST, and copy-memory tasks against tanh RNN, GRU, orthogonal RNN, and a diagonal SSM at matched parameter count. Log gradient norms, hidden-state norms, validation loss, and long-range recall accuracy. A successful result is stable gradients and better recall at long sequence lengths without extra FLOPs. Also evaluate the transfer matrix on z=e^{i theta}; its singular values should remain near one, while evaluations at |z|<1 should have singular values no larger than one.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.