Integrated-Growth Hopf Delay Scheduler
Source paper: On the slow passage through a Hopf in generalized Shishkova systems: Exponential asymptotics and maximal delay arXiv:2608.28426 ⓘ · analyzed Sep 2, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace an instantaneous largest-eigenvalue learning-rate ceiling with a delayed-instability monitor for a slowly ramped optimizer or network gain. When a dominant complex eigenpair crosses from negative to positive real part, permit a controlled post-crossing interval, but stop or roll back when the accumulated positive growth budget exceeds the perturbation/noise margin. This exploits slow-passage delay without allowing unbounded training instability.
Formulas
Mathematical statement
Near a slow Hopf crossing, represent the dominant two-dimensional mode by the complex amplitude equation $\epsilon dz/d\mu=\lambda(\mu)z$, where $\mu$ is a slowly varying control parameter, $\epsilon>0$ is the parameter sweep rate, and $\lambda(\mu)=\alpha(\mu)+i\beta(\mu)$ is the complex eigenvalue with $\alpha(0)=0$, $\alpha'(0)>0$, and $\beta(0)\neq0$. Ignoring nonlinear saturation until exit, the amplitude obeys $|z(\mu)|=|z(\mu_0)|\exp[\epsilon^{-1}\int_{\mu_0}^{\mu}\alpha(s)ds]$. Therefore the instantaneous crossing $\alpha(\mu)=0$ is not the exit point. If $r_0$ is the perturbation amplitude and $r_{\mathrm{max}}$ is the unsafe amplitude, the predicted exit parameter satisfies $\int_{\mu_0}^{\mu_{\mathrm{exit}}}\alpha(s)ds=\epsilon\log(r_{\mathrm{max}}/r_0)$. The slow manifolds may differ by an exponentially small amount $\exp(-C/\epsilon)$, with $C>0$, so noise and roundoff can determine the realized exit when that splitting is below the perturbation floor. In a neural network, $\alpha$ is estimated as the real part of a dominant complex eigenvalue of the local update Jacobian.
Implementation notes
Integrate this at the optimizer or inference-controller level for a model whose effective update Jacobian changes slowly. A practical first target is a small RNN, state-space model, or MLP trained while the learning rate is ramped. Let the control variable be $\mu=\eta$ or a normalized training-time ramp, and estimate the Jacobian of one update map on a fixed probe batch. Every $K$ steps, use Jacobian-vector products and Arnoldi iteration on a low-dimensional Krylov subspace, or finite differences for a small model, to estimate the leading conjugate pair and set $\hat\alpha_t=\operatorname{Re}\hat\lambda_t$. Maintain $B_{t+1}=B_t+\Delta\mu\hat\alpha_t$. Before the crossing, reset $B$ whenever the monitored eigenmode changes. After $\hat\alpha_t$ becomes positive, continue the ramp only while $B_t<\epsilon\log(r_{\mathrm{max}}/r_0)$. Estimate $r_0$ from minibatch-gradient noise or an injected probe perturbation, estimate $r_{\mathrm{max}}$ from the largest perturbation that remains bounded over a calibration window, and estimate $\epsilon$ from the slope of the parameter ramp per update. If the budget is exceeded, freeze the ramp, halve the learning rate, or roll back the last update. The paper mechanism is the slow-passage amplitude law and exponentially small attracting-repelling manifold splitting; Jacobians, noise levels, and thresholds are estimated empirically. First run equal-compute CIFAR-10 training of a small ResNet or sequence benchmark training of an RNN with five linear ramp rates. Compare standard instantaneous spectral clipping, this integrated-growth scheduler, and an ablation with $B=0$ that disallows delay. Pre-register the signature: measured exit should satisfy $B_{\mathrm{exit}}\approx\epsilon\log(r_{\mathrm{max}}/r_0)$ and collapse across ramp rates when plotted against $B/\epsilon$. Reject the transfer if the collapse has more than 20% relative error, divergence regularly occurs before the predicted budget, or the proposed scheduler provides no larger stable post-crossing area than instantaneous clipping.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: Inconclusive
Stage 1 — Mechanism check agent confidence 8/10
Built a reproducible complex Hopf-mode simulator and integrated-growth scheduler. The amplitude law matched direct propagation to 1.5e-14–5.8e-14 relative error, and budget-limited exits closely matched analytic predictions across three ramp rates while allowing substantial post-crossing passage versus instantaneous clipping. This validates the claimed toy slow-passage phenomenon, but not its effectiveness in neural-network training.
- Agent confidence
- 8/10
- Baseline
- Instantaneous clipping stops at mu=0, with no post-crossing interval.
- Idea
- For epsilon={0.02,0.01,0.005}, predicted exit mu={0.65894,0.58490,0.54411}; scheduler exit mu={0.65893,0.58487,0.54410}, with amplitudes approximately 0.10 at exit. Ten noisy-monitor trials at epsilon=0.01 exited at mu 0.5837–0.5885 with amplitudes 0.093–0.124.
Limitations: Only the authoritative linear two-dimensional Hopf toy system was tested. No RNN/MLP training, optimizer update-Jacobian or Arnoldi estimation, CIFAR/sequence benchmark, compute comparison, nonlinear saturation, rollback behavior, or exponentially small manifold splitting was evaluated.
How to run: python3 hopf_delay_experiment.py
Stage 2 — Benchmark vs. tuned baseline
No significant difference vs. the tuned baseline (+12.5%, p=0.49, wins 3 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0451
- Idea mean
- 0.0507
- Effect (Δ)
- 0.0056 (+12.5%; negative = idea better)
- Wins
- 3 / 8 paired seeds
- p-value
- 0.49 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 3 configs
Only the fixed 400/200-sample, 6-epoch dynamics benchmark was tested. The Hessian power-iteration real-mode proxy did not detect a positive crossing, so no post-crossing interval occurred; complex conjugate eigenpair/Arnoldi estimation, larger models, longer training, rollback behavior, nonlinear saturation, and other benchmark tracks were not tested. The generated report contains a non-finite relative_budget_error for the unobserved exit in Python JSON notation; this reflects absence of a crossing rather than a fabricated measurement.
How to run: python3 stage2_bench.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
Implemented the Integrated-Growth Hopf Delay Scheduler as a local optimizer training-loop modification on the structurally matched dynamics track, using the shared rnn_small architecture and paired seeds. The baseline achieved test MSE 0.04506 versus 0.05071 for the best idea setting, with paired delta +0.00564 and permutation p=0.49005, so there was no significant win. The trained-model signature found no positive-alpha crossing and therefore did not confirm the predicted delayed-growth mechanism at neural-network scale.
- Baseline
- dynamics/rnn_small, best lr=0.006; 8-seed test MSE mean=0.0450645, std=0.0181213
- Idea
- best idea lr=0.006, budget_scale=0.7/1.0/1.3 tied; 8-seed test MSE mean=0.0507089, std=0.00859622; paired delta=+0.00564442, p=0.49005, wins=3/8
Benchmark result
No significant difference vs. the tuned baseline (+12.5%, p=0.49, wins 3 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0451
- Idea mean
- 0.0507
- Effect (Δ)
- 0.0056 (+12.5%; negative = idea better)
- Wins
- 3 / 8 paired seeds
- p-value
- 0.49 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 3 configs
Only the fixed 400/200-sample, 6-epoch dynamics benchmark was tested. The Hessian power-iteration real-mode proxy did not detect a positive crossing, so no post-crossing interval occurred; complex conjugate eigenpair/Arnoldi estimation, larger models, longer training, rollback behavior, nonlinear saturation, and other benchmark tracks were not tested. The generated report contains a non-finite relative_budget_error for the unobserved exit in Python JSON notation; this reflects absence of a crossing rather than a fabricated measurement.
How to run: python3 stage2_bench.py
Artifacts
- bench_report.json 4.8 KB View Raw JSON
- hopf_delay_experiment.py 4.7 KB View Raw JSON
- report.md 1.4 KB View
- report_bench_2026-09-02T121352.md 4.0 KB View
- results.json 5.4 KB View Raw JSON
- stage2_bench.py 6.2 KB View Raw JSON