# Phase-Margin Residual Jacobians - ID: 2193 - Canonical URL: https://synthcore.org/idea/2193/phase-margin-residual-jacobians - API JSON: https://synthcore.org/api/idea/2193.json - API Markdown: https://synthcore.org/api/idea/2193.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2608.12591](https://arxiv.org/abs/2608.12591) - Category: regularization - Solves: stability, accuracy - ML areas: training-dynamics, regularization, optimizer, mlp, transformer - Math tags: dynamical-systems, control-theory, spectral-theory, linear-algebra, optimization - Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10 ## Idea description Use the theta-SRG of each residual-block Jacobian to regularize its gain and phase spread, rather than constraining only its spectral norm. For an implicit or deeply unrolled residual network, maintain a positive distance between the SRG enclosure of the block composition and the critical feedback point -1, giving a directly testable invertibility margin for long-horizon propagation. ## Mathematical statement For a complex matrix C and phase center theta, define the phase spread certificate Gamma_theta(C)=arcsin(min_{gamma>0} ||gamma exp(-j theta) C-I||_2), valid when Gamma_theta(C)0 such that ||gamma exp(-j theta)C-I||_2<=sin(delta), and that the spectrum of C is contained in SRG_theta(C). For block Jacobians J_i, estimate z_i=min_{gamma>0}||gamma exp(-j theta_i)J_i-I||_2 and Gamma_i=arcsin(z_i). For a product P=J_N...J_1, the arc composition rule gives SRG_theta(P) contained in the product of the individual SRG sets, with theta=sum_i theta_i. Require the resulting enclosure to avoid -1 with margin m=dist(-1, product_i SRG_hat_theta_i(J_i))>0. ## Key formulas - $$\Gamma_{\theta}(C)=\arcsin\left(\min_{\gamma>0}\left\|\gamma e^{-j\theta}C-I\right\|_2\right),\qquad \Gamma_{\theta}(C)\leq\delta\Longleftrightarrow\min_{\gamma>0}\left\|\gamma e^{-j\theta}C-I\right\|_2\leq\sin\delta.$$ - $$\mathrm{SRG}_{\theta}(AB)\subseteq \mathrm{SRG}_{\theta_1}(A)\,\mathrm{SRG}_{\theta_2}(B),\qquad \theta=\theta_1+\theta_2,$$ - $$I+P\text{ is nonsingular if }-1\notin\prod_{i=1}^{N}\mathrm{SRG}_{\theta_i}(J_i),\qquad P=J_N\cdots J_1.$$ - $$\mathcal{L}_{\mathrm{phase}}=\sum_i\left[\max(0,\Gamma_i-\Gamma_{\max})\right]^2+\lambda_m\left[\max(0,m_{\min}-m)\right]^2.$$ ## Implementation notes Integrate this into residual MLPs, transformer residual streams, or unrolled optimization networks after each block or after groups of blocks. Given a minibatch input, compute Jacobian-vector products and vector-Jacobian products with automatic differentiation; use 8 to 16 randomized Lanczos or power-iteration directions to estimate the spectral norm of gamma exp(-j theta)J-I. Search theta on a coarse grid, such as 32 angles in [0,pi), and optimize positive gamma by a one-dimensional log-space search. Pseudocode: sample directions v_k; estimate ||gamma exp(-j theta)Jv_k-v_k||; minimize over gamma and theta; accumulate Gamma_i and phase centers; propagate a conservative product SRG enclosure; add the phase loss or reduce the optimizer step size if the margin falls below m_min. The SRG certificate and product exclusion theorem come from the paper; Jacobian sampling, grid discretization, and margin estimation are empirical approximations. First test a 50-layer residual MLP on CIFAR-10 against vanilla training and spectral normalization. Measure sigma_min(I+P), gradient norms across depth, and the estimated distance from -1. The prediction is a sharp failure boundary: when the estimated SRG product first crosses -1, sigma_min(I+P) should collapse and gradients should become oscillatory or explosive. Runs maintaining m>0 should preserve bounded gradients. The estimated crossing should predict empirical invertibility failure within 20 percent. ## Verification - Status: mech_ok_no_baseline - Mechanism evidence: yes - Mechanism confirmed: yes - Practical verdict: inconclusive - Verdict: Built a deterministic phase-margin residual-Jacobian MVP using exact planar-rotation blocks. The core predictions were confirmed: certificate phase error, additive phase composition, and gain/margin singularity errors were all below 3e-15; the observed invertibility crossing was gain 1.0 as predicted, and power-growth slopes changed sign at gain 1. Adding a phase-margin hinge improved the synthetic baseline from sigma_min(I+P)=7.22e-16 to 0.2129, demonstrating the proposed mechanism, but this is not evidence of a practical ML training win. ### Mechanism check - Verdict: Built a deterministic phase-margin residual-Jacobian MVP using exact planar-rotation blocks. The core predictions were confirmed: certificate phase error, additive phase composition, and gain/margin singularity errors were all below 3e-15; the observed invertibility crossing was gain 1.0 as predicted, and power-growth slopes changed sign at gain 1. Adding a phase-margin hinge improved the synthetic baseline from sigma_min(I+P)=7.22e-16 to 0.2129, demonstrating the proposed mechanism, but this is not evidence of a practical ML training win. - Confidence: 8/10 - Limitations: Only exact 2D scaled-rotation Jacobians were tested. No nonlinear residual MLP, CIFAR-10 task, automatic-differentiation JVP/VJP estimator, randomized Lanczos approximation, spectral-normalization baseline, runtime/FLOP comparison, or general nonnormal matrix test was performed. ### Practical benchmark - Paired seeds: 8 - Baseline mean: 0 - Idea mean: 0 - p-value: 0.2886 - Paired wins: 6/8 - Benchmark verdict: no significant win ## Artifacts - [README.md](https://synthcore.org/code/875/README.md) - [bench_report.json](https://synthcore.org/code/875/bench_report.json) - [phase_margin_experiment.py](https://synthcore.org/code/875/phase_margin_experiment.py) - [report.md](https://synthcore.org/code/875/report.md) - [report_bench_2026-09-01T132416.md](https://synthcore.org/code/875/report_bench_2026-09-01T132416.md) - [results.json](https://synthcore.org/code/875/results.json) - [stage2_bench.py](https://synthcore.org/code/875/stage2_bench.py) - [Download all files as ZIP](https://synthcore.org/download/875) ## Disclaimer AI-generated research hypothesis, automatically tested. Not peer-reviewed.