✗ Failed on benchmark 2026

Free-Loss Jacobian Spectral Target

Implementation & benchmark of arXiv:2607.05693 — Free Multiplicative Convolution and Erlang Moments in Monitored Quantum Transport

Usefulness6/10

Difficulty6/10

Novelty7/10

Regularization Stability Accuracy Generalization Training dynamics Initialization Regularization Transformer Probability Spectral theory Random matrix Linear algebra

Source paper: Free Multiplicative Convolution and Erlang Moments in Monitored Quantum Transport arXiv:2607.05693 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Regularize the end-to-end Jacobian singular-value distribution of a deep network toward the explicit free small-loss law generated by independently mixed projection-like layers. The target controls several gradient-spectrum moments, including the predicted fraction of nearly preserved directions, instead of controlling only the average gradient norm.

Formulas

$$B_L=(PS_L)(PS_{L-1})\cdots(PS_1),\qquad \nu_c=(1-c)\delta_1+c\delta_0,$$

$$S_{\mu_\tau}(z)=\exp\left(\frac{\tau}{1+z}\right),\qquad \mu_\tau(\{1\})=(1-\tau)_+,$$

$$m_p(\tau)=\frac{1}{p}[w^{p-1}](1+w)^p\exp\left(-\frac{p\tau}{1+w}\right),\qquad m_1=e^{-\tau},\quad m_2=e^{-2\tau}(1+\tau),$$

$$\mathcal L_{\mathrm{spec}}=\sum_{p=1}^{K}\alpha_p\left(\log(\widehat m_p+\epsilon)-\log(m_p(\tau)+\epsilon)\right)^2,\qquad \widehat m_p=\frac{1}{R}\sum_{r=1}^{R}v_r^\top(J^\top J)^p v_r.$$

Mathematical statement

For independent Haar unitary matrices S_i in U(N) and a rank-(1-c)N orthogonal projection P, the paper studies B_L=(PS_L)(PS_{L-1})\cdots(PS_1), whose squared singular-value distribution converges for fixed L to
u_c^{\boxtimes L}, where
u_c=(1-c)\delta_1+c\delta_0, \delta_x is a point mass at x, and \boxtimes is free multiplicative convolution. In the weak-loss scaling c=1/N and L/N o au, the limiting law \mu_ au has S-transform S_{\mu_ au}(z)=\exp( au/(1+z)). Its p-th moment is m_p( au)=\frac{1}{p}[w^{p-1}](1+w)^p\exp(-p au/(1+w)), where [w^{p-1}] extracts the coefficient of w^{p-1}. Here p is a positive integer, au is effective depth divided by width, and m_p is the expected p-th moment of squared singular values. The first moments are m_1=e^{- au} and m_2=e^{-2 au}(1+ au). The law also has an atom \mu_ au(\{1\})=(1- au)_+, where (x)_+=\max(x,0). Use these moments as targets for the eigenvalues of a neural-network Jacobian Gram matrix J^ op J, with J the input-output Jacobian and N the probe dimension or hidden width. The free-law assumptions are exact only for independent isotropic mixing and weak loss, so this is an experimentally testable initialization or regularization hypothesis rather than a direct theorem about trained networks.

Implementation notes

Integrate the method into the training loop of a deep MLP, residual transformer, or state-space stack. Let J denote the Jacobian from a minibatch input representation x to its output representation y; never materialize J. Every T training steps, draw R normalized Gaussian or Rademacher probe vectors v_r with the same dimension as x. For each probe set u_{r,0}=v_r, then repeatedly compute u_{r,k+1}=J^ op(Ju_{r,k}) using a Jacobian-vector product followed by a vector-Jacobian product. Estimate each Gram moment by \widehat m_p=(1/R)\sum_r v_r^ op u_{r,p}, for p=1,...,K, with K=2 or 3. Select an effective parameter au, initially L/N where L is the number of mixing blocks and N is their width, and sweep au in {0.25,0.5,1,2}. Precompute m_p( au) from the coefficient-extraction formula using polynomial expansion or automatic symbolic differentiation. Add \lambda_{spec}\mathcal L_{spec} to the task loss, stop-gradient through m_p, clip the auxiliary loss, and apply it only during the first 10-30% of training or every T steps. The mathematical quantities computed exactly are the target moments; \widehat m_p is estimated empirically by Hutchinson-style probes. First test a 12-layer width-256 MLP on CIFAR-10 or a 6-layer transformer on a small language-model corpus. Compare Xavier or orthogonal initialization, a scalar Jacobian-norm penalty, and this spectral penalty. Record early loss descent at equal FLOPs, gradient-norm variance, activation variance, final accuracy or perplexity, and singular-value histograms. Success requires closer tracking of m_1 and m_2, reduced exploding or vanishing gradients, and faster early optimization without worse final task quality.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible free-loss moment calculator, Haar projection-product verification, and MLP spectral-regularization experiment. The product check closely matched m1-m3 and reproduced the atom masses: empirical versus target moment errors were below 0.005 for tau in {0.25, 0.5, 1, 2}. In the MLP, spectral regularization reduced final log-moment RMSE from 4.15 to 2.82 and reduced both moments substantially toward target, but final task MSE worsened from 0.767 to 0.922 and optimization was slower; therefore this is a genuine spectral-control signal, not evidence of an end-to-end task-performance win.

Agent confidence: 8/10

Baseline: Haar check: target/empirical moment absolute errors for tau=.25,.5,1,2 were respectively [0.00095,0.00039,0.00019], [0.00497,0.00378,0.00291], [0.00463,0.00380,0.00339], [0.00338,0.00194,0.00126]. MLP final MSE=0.7673; final moments=[5.3055,234.4828]; log-moment RMSE=4.151.
Idea: Haar check reproduced atom targets 0.75,0.5,0,0. MLP final MSE=0.9215; final moments=[2.1802,42.7845]; log-moment RMSE=2.8229, a 32% reduction versus baseline, but with worse task loss.

Limitations: The MLP experiment used exact per-example Jacobians rather than memory-efficient JVP/VJP probes, only one seed and one tiny regression task, 120 steps, K=2, and no CIFAR/transformer benchmark. The implementation did not test GPU execution, multiple regularizer weights, equal-FLOP comparisons, gradient variance, or trained-network singular-value histograms.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.05693) Sep 4, 2026

No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 21.8103
Idea mean: 21.8103
Effect (Δ): 0 (−-0.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±0.0%
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

Only the structurally matched built-in tabular track was tested. The intervention used a small exact Jacobian probe loss during early training; vision, sequence, dynamics, FLOP accounting, gradient variance, and singular-value histograms were not tested.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Failed ✗

Agent confidence: 10/10

The official registered tabular/mlp_tiny benchmark was run through bench.make_report() with a tuned baseline sweep and 8 paired seeds. The spectral target had identical mean test MSE to baseline, delta_mean=0 and permutation p=1.0, so there was no significant task-metric win. The projection-product math check passed, but the trained-network mechanism signature was not confirmed because observed moments differed substantially from the free-law targets.

Baseline: Mean test MSE 21.810271620750427; best lr=0.003, epochs=8.
Idea: Mean test MSE 21.810271620750427; best lr=0.003, lambda=0.0001, epochs=8; paired delta=0.0; p=1.0.

Benchmark result