△ Mechanism confirmed, baseline not beaten 2026

Phase-Margin Residual Jacobians

Implementation & benchmark of arXiv:2608.12591 — The $θ$-Symmetric SRG with Applications to Stability of Cactus Dynamic Networks

Usefulness7/10

Difficulty5/10

Novelty7/10

Regularization Stability Accuracy Training dynamics Regularization Optimizer Mlp Transformer Dynamical systems Control theory Spectral theory Linear algebra Optimization

Source paper: The $θ$-Symmetric SRG with Applications to Stability of Cactus Dynamic Networks arXiv:2608.12591 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the theta-SRG of each residual-block Jacobian to regularize its gain and phase spread, rather than constraining only its spectral norm. For an implicit or deeply unrolled residual network, maintain a positive distance between the SRG enclosure of the block composition and the critical feedback point -1, giving a directly testable invertibility margin for long-horizon propagation.

Formulas

$$\Gamma_{\theta}(C)=\arcsin\left(\min_{\gamma>0}\left\|\gamma e^{-j\theta}C-I\right\|_2\right),\qquad \Gamma_{\theta}(C)\leq\delta\Longleftrightarrow\min_{\gamma>0}\left\|\gamma e^{-j\theta}C-I\right\|_2\leq\sin\delta.$$

$$\mathrm{SRG}_{\theta}(AB)\subseteq \mathrm{SRG}_{\theta_1}(A)\,\mathrm{SRG}_{\theta_2}(B),\qquad \theta=\theta_1+\theta_2,$$

$$I+P\text{ is nonsingular if }-1\notin\prod_{i=1}^{N}\mathrm{SRG}_{\theta_i}(J_i),\qquad P=J_N\cdots J_1.$$

$$\mathcal{L}_{\mathrm{phase}}=\sum_i\left[\max(0,\Gamma_i-\Gamma_{\max})\right]^2+\lambda_m\left[\max(0,m_{\min}-m)\right]^2.$$

Mathematical statement

For a complex matrix C and phase center theta, define the phase spread certificate Gamma_theta(C)=arcsin(min_{gamma>0} ||gamma exp(-j theta) C-I||_2), valid when Gamma_theta(C)<pi/2. The paper proves Gamma_theta(C)<=delta if and only if there exists gamma>0 such that ||gamma exp(-j theta)C-I||_2<=sin(delta), and that the spectrum of C is contained in SRG_theta(C). For block Jacobians J_i, estimate z_i=min_{gamma>0}||gamma exp(-j theta_i)J_i-I||_2 and Gamma_i=arcsin(z_i). For a product P=J_N...J_1, the arc composition rule gives SRG_theta(P) contained in the product of the individual SRG sets, with theta=sum_i theta_i. Require the resulting enclosure to avoid -1 with margin m=dist(-1, product_i SRG_hat_theta_i(J_i))>0.

Implementation notes

Integrate this into residual MLPs, transformer residual streams, or unrolled optimization networks after each block or after groups of blocks. Given a minibatch input, compute Jacobian-vector products and vector-Jacobian products with automatic differentiation; use 8 to 16 randomized Lanczos or power-iteration directions to estimate the spectral norm of gamma exp(-j theta)J-I. Search theta on a coarse grid, such as 32 angles in [0,pi), and optimize positive gamma by a one-dimensional log-space search. Pseudocode: sample directions v_k; estimate ||gamma exp(-j theta)Jv_k-v_k||; minimize over gamma and theta; accumulate Gamma_i and phase centers; propagate a conservative product SRG enclosure; add the phase loss or reduce the optimizer step size if the margin falls below m_min. The SRG certificate and product exclusion theorem come from the paper; Jacobian sampling, grid discretization, and margin estimation are empirical approximations. First test a 50-layer residual MLP on CIFAR-10 against vanilla training and spectral normalization. Measure sigma_min(I+P), gradient norms across depth, and the estimated distance from -1. The prediction is a sharp failure boundary: when the estimated SRG product first crosses -1, sigma_min(I+P) should collapse and gradients should become oscillatory or explosive. Runs maintaining m>0 should preserve bounded gradients. The estimated crossing should predict empirical invertibility failure within 20 percent.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a deterministic phase-margin residual-Jacobian MVP using exact planar-rotation blocks. The core predictions were confirmed: certificate phase error, additive phase composition, and gain/margin singularity errors were all below 3e-15; the observed invertibility crossing was gain 1.0 as predicted, and power-growth slopes changed sign at gain 1. Adding a phase-margin hinge improved the synthetic baseline from sigma_min(I+P)=7.22e-16 to 0.2129, demonstrating the proposed mechanism, but this is not evidence of a practical ML training win.

Agent confidence: 8/10

Baseline: Synthetic vanilla optimization: sigma_min(I+P)=7.22e-16, margin=1.22e-16
Idea: Phase-margin hinge: sigma_min(I+P)=0.212911, margin=0.212911; certificate error=3.33e-16; composition error=4.44e-16; crossing observed/predicted=1.000/1.000

Limitations: Only exact 2D scaled-rotation Jacobians were tested. No nonlinear residual MLP, CIFAR-10 task, automatic-differentiation JVP/VJP estimator, randomized Lanczos approximation, spectral-normalization baseline, runtime/FLOP comparison, or general nonnormal matrix test was performed.

How to run: python3 phase_margin_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.12591) Sep 2, 2026

Ahead of the baseline (−3.3%) but not statistically significant (p=0.2886, wins 6 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0463
Idea mean: 0.0448
Effect (Δ): -0.0015 (−3.3%; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.2886 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±7.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the small registered dynamics track, 400 training samples, 200 test samples, 12 epochs, and a four-block residual RNN were tested. The Jacobian certificate used two coordinate probes and a coarse gamma grid; no large-scale network, wall-clock overhead, full product SRG enclosure, or broader architecture study was run.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

The phase-margin Jacobian penalty was implemented and evaluated on the structurally matched registered dynamics track using the shared residual RNN architecture. It reduced mean MSE from 0.0463165 to 0.0447833, but the paired permutation p-value was 0.2886, so the required significant-win criterion was not met.

Baseline: 8-seed tuned mean MSE 0.04631647397764027, best lr=0.001, epochs=12
Delta mean: -0.001533161848783493
Idea: 8-seed best mean MSE 0.04478331212885678, lr=0.001, epochs=12
P value: 0.2886

Benchmark result