✗ Mechanism failed 2026

Critical-Gain Covariance Controller

Implementation & benchmark of arXiv:2608.00306 — Mechanistic bridges from receptors to whole-brain dynamics: mean-field reductions, validity domains, and computational trade-offs

Usefulness7/10

Difficulty5/10

Novelty6/10

Dynamics Stability Accuracy Generalization Rnn Ssm Training dynamics Regularization Dynamical systems Spectral theory Linear algebra Stochastic processes

Source paper: Mechanistic bridges from receptors to whole-brain dynamics: mean-field reductions, validity domains, and computational trade-offs arXiv:2608.00306 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Track the covariance of a small recurrent population state and regulate its effective gain before finite-size fluctuations diverge. The controller uses the covariance Jacobian eigenvalues from the paper, making the distance to criticality an explicit adaptive regularization signal for recurrent or state-space neural networks.

Formulas

$$\frac{d}{dt}\begin{pmatrix}c_{EE}\\c_{II}\\c_{EI}\end{pmatrix}=\underbrace{\begin{pmatrix}2k_{E}-2&0&2k_{I}\\0&2k_{I}-2&2k_{E}\\k_{E}&k_{I}&k_{E}+k_{I}-2\end{pmatrix}}_{J}\begin{pmatrix}c_{EE}\\c_{II}\\c_{EI}\end{pmatrix}+\underbrace{\begin{pmatrix}\frac{m_{0}(1/T-m_{0})}{N_{exc}}\\\frac{m_{0}(1/T-m_{0})}{N_{inh}}\\0\end{pmatrix}}_{D}$$

$$\lambda_{3}=-2,\qquad \lambda_{4}=2(\Delta-1),\qquad \lambda_{5}=\Delta-2,\qquad \Delta=k_E-k_I$$

$$Jc^{FP}=-D,\qquad c^{FP}=-J^{-1}D$$

$$\mathcal L_{\mathrm{crit}}=\alpha\,\big[\max(0,\Delta-(1-\varepsilon))\big]^2+\beta\,\log\!\left(1+\frac{\widehat{\operatorname{tr}C}}{\tau}\right)$$

Mathematical statement

In the paper's two-population reduction, $c=(c_{EE},c_{II},c_{EI})^\top$ contains excitatory variance, inhibitory variance, and cross-covariance; $k_E$ and $k_I$ are effective excitatory and inhibitory gains, $\Delta=k_E-k_I$, and $m_0$, $T$, $N_{exc}$, $N_{inh}$ define the baseline rate, bin duration, and population sizes. The covariance dynamics are an affine linear system $\dot c=Jc+D$, with the displayed $J$ and $D$. Its eigenvalues are $\lambda_3=-2$, $\lambda_4=2(\Delta-1)$, and $\lambda_5=\Delta-2$. The fixed point satisfies $Jc^{FP}=-D$, and the paper reports fluctuations growing near $\Delta=1$, with order $(1-\Delta)^{-3}$ scaling. We transfer the eigenvalue margin $1-\Delta$ as a trainable stability statistic, not as a claim that arbitrary neural networks exactly obey the two-population model.

Implementation notes

Integrate the controller into a recurrent block or linear state-space layer after the recurrent weight matrix has been applied and before the nonlinearity. Partition hidden channels into two groups, interpreted as E and I, and parameterize recurrent weights as $W=\begin{bmatrix}W_{EE}&W_{EI}\\W_{IE}&W_{II}\end{bmatrix}$. Estimate effective gains $k_E,k_I$ every training step or every 100 steps using one power iteration per block: $k_E\approx\|W_{EE}v\|_2/\|v\|_2$ and $k_I\approx\|W_{II}u\|_2/\|u\|_2$, with nonnegative gains enforced by softplus if desired. Compute $\Delta=k_E-k_I$, and add the criticality penalty $\alpha[\max(0,\Delta-(1-\varepsilon))]^2$. In parallel, estimate the empirical hidden covariance $\widehat C=(H-\bar H)^\top(H-\bar H)/(B-1)$ over batch and time samples, and penalize excessive trace or largest eigenvalue. The paper's exact $J,D$, eigenvalues, and fixed-point relation define the two-population diagnostic; the general-network covariance is estimated empirically. For a more faithful controller, integrate $c_{t+1}=c_t+\delta(Jc_t+D)$ and compare it with observed three-component covariance. Start with a 1-2 layer GRU on sequential MNIST or WikiText-2 against an unconstrained GRU and spectral normalization. Log $\Delta$, covariance trace, gradient norm, and long-horizon loss. Success is elimination of exploding hidden covariance and improved loss over long sequences with less performance degradation than hard spectral clipping.

Verification

Mechanism failed

Stage 1 · Toy mechanism gate: Failed ✗

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 10/10

Built the exact covariance Jacobian diagnostic, fixed-point sweep, and a small CUDA RNN comparison with the proposed gain/covariance penalty. The stated Delta eigenvalue formula failed with maximum absolute error 3.2; symbolic and numerical checks instead confirmed eigenvalues {-2, 2(kE+kI-1), kE+kI-2} to 4.0e-12, with the stability boundary at kE+kI=1. The measured decay-rate scaling matched the corrected prediction closely and fixed-point log-log scaling was -0.945 versus -1.0, but the proposed controller did not reduce hidden covariance and only modestly improved long loss (0.7053 vs 0.7456).

Agent confidence: 10/10

Baseline: long_loss=0.745608; long_trace=15.7703; long_delta=-0.8958
Idea: long_loss=0.705277; long_trace=15.7369; long_delta=-0.8989; stated-eigenvalue max error=3.2; corrected-eigenvalue max error=4.04e-12; corrected decay scaling measured/predicted near 1.0; fixed-point slope=-0.945/-1 predicted

Limitations: Only a tiny synthetic next-symbol RNN was tested, not sequential MNIST, language modeling, GRU/SSM architectures, spectral normalization, or multi-seed statistical comparisons. The supplied paper formula was not reconciled against a broader model variant because the displayed J directly contradicts the stated Delta eigenvalues.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 3 files · code, reports and structured results

🐍 experiment.py 6.0 KB View Raw
📄 report.md 1.5 KB View
📊 results.json 4.1 KB View Raw