△ Mechanism confirmed, baseline not beaten 2026

Rank-One Feedback Spectrum Regularizer

Implementation & benchmark of arXiv:2607.02877 — Endogenous Feedback in Size-Structured Transport Equations

Usefulness6/10

Difficulty6/10

Novelty6/10

Dynamics Stability Generalization Accuracy Ssm Rnn Training dynamics Regularization Dynamical systems Control theory Linear algebra

Source paper: Endogenous Feedback in Size-Structured Transport Equations arXiv:2607.02877 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Model the scalar feedback route in a recurrent layer as a rank-one perturbation of its open-loop transition. Regularize the frequency response of that route so that no mode reaches unit loop gain, directly targeting oscillatory and slowly decaying instabilities rather than relying only on gradient clipping.

Formulas

$$\mathcal E(\lambda)=1,$$

$$\mathcal E(0)=\Phi'(E^*)=B(0),$$

$$\delta x_{k+1}=A\,\delta x_k+b\,\delta E_k,\qquad \delta E_k=c^\top\delta x_k,\qquad J=A+bc^\top,$$

$$\det(zI-J)=\det(zI-A)\left[1-c^\top(zI-A)^{-1}b\right],\qquad \mathcal E(z):=c^\top(zI-A)^{-1}b.$$

Mathematical statement

The paper linearizes around a stationary solution and obtains a finite-memory renewal equation with characteristic equation E(λ)=1; the roots determine the feedback spectrum and stability. It also states the zero-discount identity E(0)=Φ′(E*)=B(0), linking closure resonance to loop gain. For a neural linearization, let δx_{k+1}=Aδx_k+bδE_k and δE_k=cᵀδx_k, where A is the open-loop Jacobian, b is the state response to scalar feedback, c is the scalar readout, and J=A+bcᵀ is the closed-loop Jacobian. The matrix determinant lemma gives det(zI−J)=det(zI−A)[1−cᵀ(zI−A)⁻¹b]. Therefore closed-loop modes satisfy the transferred characteristic condition E(z):=cᵀ(zI−A)⁻¹b=1. Keeping |E(z)| below one on a contour around the unit circle is a practical sufficient margin, though not a complete stability theorem for arbitrary nonnormal A.

Implementation notes

(1) Add this to the transition core of an RNN or diagonal-plus-low-rank SSM. During training, obtain A, b, and c at minibatch states using automatic differentiation: A is the Jacobian with respect to the hidden state, b is the derivative of the next state with respect to the scalar feedback, and c is the derivative of the feedback with respect to the state. Do not materialize J. (2) For probe points z=rho exp(i theta), solve (z*I-A) q=b, compute H=dot(c,q), and accumulate penalty += relu(abs(H)-(1-margin))**2. Use 8 to 16 theta values and two or three radii near the unit circle; detach the penalty for most minibatches if cost is excessive. (3) The mathematical quantity is H=E(z)=cᵀ(zI−A)⁻¹b. Estimate it with linear solves, while independently materializing J on a small model to compare the predicted gain with its largest eigenvalue and impulse-response decay. At z=1, estimate the zero-frequency loop gain cᵀ(I−A)⁻¹b when the solve is well conditioned. (4) First test a 64-dimensional SSM on sequential MNIST and permuted sequential MNIST against the same model without the penalty, gradient clipping, and spectral normalization. Measure long-context accuracy, gradient norms, impulse-response decay, and training loss oscillations at equal FLOPs. The expected signal is reduced oscillation and improved stability near the largest usable step size without hurting memory-task accuracy. Falsify the idea if the frequency response does not predict observed instability or if the regularizer consistently damages task performance.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible rank-one feedback spectrum regularizer using complex linear solves over a contour, plus an independent determinant-lemma check. The identity held with maximum relative error 5.8e-15 and closed-loop characteristic residual 8.0e-14. In the scalar toy, the baseline reached g=0.1800 with pole 1.1000, contour gain 3.00, and 19-step impulse magnitude 6.12; the regularized model reached g=0.0650 with pole 0.9850, contour gain 1.084, and impulse magnitude 0.751, but its task MSE increased from approximately 0 to 0.0132. This is a clear stability signal, not evidence of improved task accuracy.

Agent confidence: 8/10

Baseline: g=0.1800; task MSE=9.44e-15; max contour |H|=3.0000; closed-loop pole=1.1000; |impulse[19]|=6.116
Idea: g=0.0650; task MSE=0.01322; max contour |H|=1.0839; closed-loop pole=0.9850; |impulse[19]|=0.7509; penalty=0.000705

Limitations: Only a scalar analytical toy was trained; no minibatch autodiff Jacobians, 64-dimensional RNN/SSM, sequential or permuted MNIST, gradient-clipping comparison, spectral normalization comparison, FLOP accounting, CUDA execution, or nonnormal-A stress test was performed. The contour margin was not fully achieved because the chosen toy optimum and penalty weight leave a small residual violation.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.02877) Sep 4, 2026

No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0282
Idea mean: 0.0282
Effect (Δ): 0 (−-0.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±0.0%
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

The runtime-reduced run used 300 training samples, 100 test samples, 4 epochs, and three shared learning rates. The expensive contour penalty was evaluated only on the first batch of each epoch. No larger-budget, FLOP, nonnormal-A, or largest-step-size stress test was run.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Failed ✗

Agent confidence: 9/10

The rank-one feedback spectrum regularizer was implemented and evaluated on the registered dynamics/rnn_small benchmark with paired seeds and a tuned baseline sweep. The idea and baseline tied exactly on the best shared configuration: mean test MSE 0.0281912, paired delta 0, permutation p=1.0. The trained-model mechanism signature showed a stable closed-loop radius and rapid impulse decay, but there was no task-metric improvement.

Baseline: best_cfg lr=0.008, epochs=4; full 8-seed test MSE mean=0.028191218618303537, std=0.00941179428559433
Idea: best shared cfg lr=0.008, epochs=4; 8-seed test MSE mean=0.028191218618303537, std=0.00941179428559433; paired delta=0.0; p=1.0

Benchmark result