Periodic CMV Unitary Recurrent Layer
Implementation & benchmark of arXiv:2607.06400 — Fast computation of eigenvalues of periodic CMV matrices
Source paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.
Formulas
Mathematical statement
The paper factors a CMV matrix as U=LM, where L and M are products of disjoint 2x2 unitary blocks: L=diag(C_0,C_2,C_4,...) and M=diag(C_1,C_3,C_5,...), with staggered supports so that LM is pentadiagonal rather than block diagonal. Each core C_k is a 2x2 unitary matrix. Periodicity is imposed by C_{k+n}=C_k for even state width n, so only one period of cores is stored. The paper gives the core parameterization C(c,s)=[[conjugate(c),-s],[s,c]] with |c|^2+|s|^2=1, along with exchange and rotator cores. The turnover identity refactors overlapping cores without changing their product: A_i B_{i+1} C_i = Ahat_{i+1} Bhat_i Chat_{i+1}. This local identity permits a boundary or cyclic core to move through the factor sequence while preserving the represented unitary operator. Since each factor is unitary, U is unitary and therefore ||Ux||_2=||x||_2 for every state x; this prevents recurrent-state norm explosion or collapse before nonlinearities are applied.
Implementation notes
(1) Integration point: use this transition as the recurrent matrix in an Elman RNN or as the state transition in a linear/nonlinear SSM. For a real implementation, either represent each complex coordinate with two real channels or restrict the cores to real 2D rotations. Store one parameter pair for each core in a period instead of storing an n-by-n matrix. For a real rotation, use C_k=[[cos(theta_k),-sin(theta_k)],[sin(theta_k),cos(theta_k)]]. For the complex parameterization, maintain unconstrained c_k and s_k and normalize them by r_k=sqrt(|c_k|^2+|s_k|^2+epsilon), setting c_k<-c_k/r_k and s_k<-s_k/r_k. (2) Pseudocode: input h_t, x_t, and period-core parameters; set v=h_t; for each odd-position core in M, apply its 2x2 unitary to adjacent coordinates; for each even-position core in L, apply its 2x2 unitary to the staggered adjacent coordinates; for the periodic version, apply the last core to the wrapped pair (v_n,v_1); compute h_{t+1}=phi(Uv+B x_t), or use h_{t+1}=Uv+phi(Bx_t) when strict linear norm preservation is preferred. If a cyclic core must be moved through an overlapping sequence, form the affected 3x3 product and refactor it as the turnover equation specifies using a small unitary QR or SVD, replacing the three old local cores by the three new cores. (3) Compute directly from the mathematics: the alternating support pattern, periodic parameter sharing, unitary local updates, and norm-preserving transition. Estimate empirically: hidden-state norm distributions, gradient norms, Jacobian singular values, and the best nonlinear placement or optional scalar gain. (4) First cheap experiment: train a width-256 sequential-MNIST or copy-memory model with a dense tanh RNN, a standard orthogonal RNN, and this CMV layer using periods 2, 8, 32, and 256. Record validation accuracy, loss versus sequence length, gradient norm variance, parameter count, peak memory, and wall-clock throughput. Success means stable gradients over longer sequences than the dense baseline, comparable or better accuracy than the orthogonal baseline, and approximately linear parameter and memory scaling. A stronger result would be a period-8 or period-32 model matching the full-width unitary baseline with substantially fewer trainable parameters.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a differentiable periodic CMV-style recurrent transition using alternating local real 2x2 rotation cores with period tying. The numerical check confirmed orthogonality and norm preservation to about 3.5e-6 error, while a 1.05-scaled dense orthogonal control amplified norms by 131.5x after 100 steps. However, on the toy sum-of-sequence task the CMV models reached 0.678 accuracy versus 0.775 for the dense orthogonal RNN and were about 10x slower in this unoptimized local-scatter implementation, so the claimed practical learning win was not observed.
- Agent confidence
- 9/10
- Baseline
- Dense orthogonal RNN: 1,121 parameters, final loss 0.5885, accuracy 0.7754, 4.62 s; expansive dense norm ratio after 100 steps: 131.50x
- Idea
- CMV period-8: 105 parameters, final loss 0.6252, accuracy 0.6777, 48.23 s; period-32: 129 parameters, final loss 0.6311, accuracy 0.6777, orthogonality error 2.38e-7 and norm-ratio maximum deviation 3.46e-6
Limitations: Only a small width-32, length-40 synthetic sum task was tested; no sequential-MNIST, copy-memory, complex cores, turnover refactorization, gradient/Jacobian statistics, optimized batched factor kernels, or broader random-seed evaluation was performed.
How to run: python3 run_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- report.md 1.5 KB View
- results.json 2.6 KB View Raw
- run_experiment.py 4.6 KB View Raw