△ Mechanism confirmed, baseline not beaten 2026

Covariance-Eigenmode Bifurcation Scheduler

Implementation & benchmark of arXiv:2607.10723 — Continuity and Discontinuity of McKean-Vlasov Phase Transitions via Bifurcation Theory

Usefulness7/10

Difficulty5/10

Novelty7/10

Dynamics Stability Generalization Optimizer Training dynamics Regularization Dynamical systems Bifurcations Linear algebra Statistics

Source paper: Continuity and Discontinuity of McKean-Vlasov Phase Transitions via Bifurcation Theory arXiv:2607.10723 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Run a small ensemble of neural-network replicas and treat their parameter or representation distribution as a mean-field state. Estimate the linearized replica-to-replica response and its covariance eigenmodes; when the leading mode approaches the critical eigenvalue associated with a pitchfork bifurcation, reduce the learning rate or noise, and when it is safely subcritical, increase exploration. The eigenvector identifies the parameter or feature direction in which branch splitting is expected.

Formulas

$$C=\frac{1}{N}\sum_{i=1}^{N}(z_i-m)(z_i-m)^\top,\qquad m=\frac{1}{N}\sum_{i=1}^{N}z_i.$$

$$A(\mu)=I-J(\mu),\qquad J(\mu)=\frac{\partial F}{\partial m}(m_c,\mu),\qquad \det A(\mu_c)=0.$$

$$0=\alpha(\mu-\mu_c)a-\beta a^3+O(a^5),\qquad a=v_c^\top(m-m_c),\qquad |a|\simeq\sqrt{\frac{\alpha}{\beta}(\mu-\mu_c)}.$$

$$\rho(J(\mu))<1\ \text{is locally stable},\qquad \rho(J(\mu))=1\ \text{is the predicted stability boundary}.$$

Mathematical statement

Let $z_i\in\mathbb{R}^{d}$ be the state of replica $i$, such as a selected layer's pooled representation or a low-dimensional parameter projection, and let $m=N^{-1}\sum_{i=1}^{N}z_i$ be the ensemble mean. Define the empirical covariance $C=N^{-1}\sum_i(z_i-m)(z_i-m)^\top$. Let $F(m,\mu)$ be the estimated one-step mean update under control parameter $\mu$, where $\mu$ may be learning rate, injected noise, or replica coupling. A bifurcation occurs when the Jacobian $J=\partial F/\partial m$ has an eigenvalue crossing $1$ in discrete time, equivalently when the linearized residual operator $A=I-J$ has an eigenvalue crossing $0$. The critical direction is the eigenvector $v_c$ satisfying $A v_c=0$. For a reflection-symmetric pitchfork, the scalar order parameter $a=v_c^\top(m-m_c)$ obeys $0=\alpha(\mu-\mu_c)a-\beta a^3+O(a^5)$, with nonzero $\alpha,\beta$; hence $|a|\propto|\mu-\mu_c|^{1/2}$ on the broken-symmetry branches. In implementation, $J$ is estimated by finite perturbations or Jacobian-vector products, while $C$ supplies a cheap proxy for the critical mode.

Implementation notes

1. Exact integration point: attach this monitor to ordinary SGD or Adam, using $N=8$--$32$ replicas that share minibatches but have independent dropout masks or small independent Gaussian parameter noise. At every $K$ optimizer steps, record either the final-layer representation of a fixed probe batch or a random Gaussian projection of parameters. 2. Pseudocode: compute $m$ and $C$; estimate the dominant covariance eigenvector $v_1$ by power iteration; perturb the ensemble mean by $+\epsilon v_1$ and $-\epsilon v_1$ while holding the batch fixed; run one optimizer step and estimate $Jv_1\approx[F(m+\epsilon v_1,\mu)-F(m-\epsilon v_1,\mu)]/(2\epsilon)$; set the local gain to $g=v_1^\top Jv_1$; if $g>1-\delta$, multiply the learning rate and injected noise by $0.5$, otherwise slowly increase them toward nominal values. Also record $a_t=v_1^\top(m_t-m_0)$ and fit $a^2$ against $\mu-\mu_c$. 3. Computed from the paper's mechanism: the eigenvalue-crossing criterion, critical eigenvector, and pitchfork square-root law. Estimated empirically: $J$, $\mu_c$, finite-ensemble covariance, and coefficients $\alpha,\beta$. 4. First cheap experiment: train eight small MLPs on MNIST with SGD, comparing fixed learning rate, cosine decay, and this scheduler. Sweep learning rate and dropout/noise level. The falsifiable prediction is that replica stability changes near the measured $g=1$ boundary, and that a symmetry-preserving setup shows $a^2$ approximately linear in $\mu-\mu_c$ above threshold; the fitted critical exponent should be near $1/2$ and the observed transition point should agree with the $g=1$ crossing within roughly 20%. Test whether intervention before the crossing reduces loss spikes without simply lowering the final learning rate.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable covariance-eigenmode toy verifier and scheduler prototype. The stability boundary was predicted at gamma=0.800 and measured at 0.7985 (0.18% error); covariance modes aligned with the true unstable eigenvector at >=0.9997, and observed variance-growth slopes matched 2 log(gamma*lambda) near the boundary. The pitchfork sweep recovered exponent 0.500000 versus the predicted 0.5. The covariance-driven scheduler substantially reduced stress-test state magnitude versus fixed learning rate, although this is not evidence of improved neural-network generalization.

Agent confidence: 8/10

Baseline: Fixed-rate stress test: final mean |state|=1.5514e29, maximum |state|=4.6219e29, reduced-LR fraction=0, maximum observed gain=2.1909.
Idea: Covariance scheduler: final mean |state|=1.7368e14, maximum |state|=5.5464e14, reduced-LR fraction=0.7856, maximum observed gain=1.9099. Stability sweep: predicted boundary gamma=0.8000, empirical boundary gamma=0.7985; pitchfork exponent predicted=0.5, observed=0.500000.

Limitations: Only low-dimensional deterministic/noisy toy dynamics were tested, not the planned MNIST MLP ensemble. The pitchfork normal-form experiment directly uses the assumed normal form, so it validates scaling and implementation rather than discovering the phenomenon in a neural network. The scheduler stress test enters negative curvature and has unbounded signed quadratic loss; no wall-clock, FLOP, GPU, dropout, Adam, representation-monitoring, or generalization comparison was performed.

How to run: python3 toy_verify.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.10723) Sep 2, 2026

Ahead of the baseline (−26.7%) but not statistically significant (p=0.3832, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0102
Idea mean: 0
Effect (Δ): -0.0027 (−26.7%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.3832 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±65.1%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The official registered dynamics track was run, but with reduced equal-budget settings n_train=400, n_test=200, and 6 epochs rather than the harness README defaults. The implementation uses covariance spread growth as a cheap leading-mode proxy rather than explicitly estimating a full covariance eigenvector and finite-difference Jacobian-vector product.

How to run: python3 official_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

The covariance-eigenmode scheduler was evaluated on the registered dynamics track using the official rnn_small architecture, eight paired seeds, a tuned baseline sweep, and a parity-matched idea sweep. Although the idea reduced mean MSE, the paired permutation test was not significant (p=0.3832), so the official verdict is no significant win.

Baseline: dynamics/rnn_small fixed Adam, tuned best lr=0.01; full eight-seed mean MSE 0.010180400568060577.
Idea: dynamics/rnn_small covariance scheduler, best lr=0.01; eight-seed mean MSE 0.0074651449685916305; paired delta -0.0027152555994689465; p=0.3832.

Benchmark result