✓✓ Beats tuned baseline 2026

Semi-Passive Energy-Gated Optimizer

Usefulness8/10

Difficulty4/10

Novelty7/10

Dynamics Stability Speedup Generalization Optimizer Training dynamics Scheduler Control theory Dynamical systems Stochastic processes Optimization

Source paper: Robust Semi-passive Velocity Field Control with Boundedness Guarantees for Safe Interaction between Mechanical Systems and Physical Environment arXiv:2608.30193 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Treat optimization as a forced dynamical system whose state is the parameter velocity and whose input is the minibatch gradient. Permit ordinary momentum updates below a target energy, but smoothly increase damping when optimizer energy exceeds that target. This preserves less-conservative behavior in low-energy regions while imposing dissipative dynamics during potentially divergent excursions.

Formulas

$$E(v)=\frac{1}{2}v^{\top}Mv,\qquad q(E)=\sigma\!\left(\frac{E-E_\star}{\tau}\right),\qquad \sigma(z)=\frac{1}{1+e^{-z}}.$$

$$M\dot v=-g-q(E)cMv,\qquad \dot\theta=v.$$

$$\dot E=v^{\top}M\dot v=-v^{\top}g-2c\,q(E)E.$$

$$|v^{\top}g|\leq B\ \Longrightarrow\ \dot E\leq B-2cE\ \text{when }q(E)\approx1,\qquad \limsup_{t\to\infty}E(t)\leq\frac{B}{2c}.$$

Mathematical statement

Let $\theta\in\mathbb{R}^d$ be network parameters, $v$ the optimizer velocity, $g$ the minibatch gradient, and $M\succ0$ a mass or preconditioning matrix. Define storage energy $E(v)=\frac12v^\top Mv$. The transferable semi-passivity condition is imposed only for $E>E_\star$: the energy derivative must contain a dissipative term proportional to $E-E_\star$. Use the smooth gate $q(E)=\sigma((E-E_\star)/\tau)$, where $E_\star$ is the activation threshold, $\tau>0$ is transition width, and $\sigma$ is the logistic function. For $\dot\theta=v$ and $M\dot v=-g-q(E)cMv$, with damping $c>0$, the energy derivative is $\dot E=-v^\top g-2cq(E)E$. If the forcing satisfies $|v^\top g|\le B$, then in the active regime $\dot E\le B-2cE$, yielding the ultimate bound $\limsup E\le B/(2c)$.

Implementation notes

Integrate this at the momentum-to-parameter-update interface of SGD with momentum, or after Adam bias correction. Maintain a velocity vector $v$ and choose $M=I$ for the first test. For each minibatch, compute the gradient $g$, calculate $E=0.5v^\top Mv$, calculate $q=\operatorname{sigmoid}((E-E_\star)/\tau)$, and update the velocity with $v\leftarrow v-\eta M^{-1}g-\eta qcv$; then update parameters using $\theta\leftarrow\theta+v$. With Adam, use the preconditioned bias-corrected first moment as the velocity and apply the same multiplicative damping before changing parameters. The energy, gate, gradient work $|v^\top g|$, and damping contribution are directly computed. The disturbance bound $B$ is estimated online using an exponential moving maximum of $|v^\top g|$, rather than assumed known. First test a 3-layer MLP on MNIST with momentum SGD, comparing ordinary momentum, gradient clipping, and the gated optimizer over a learning-rate sweep. Log energy and update norms. The prediction is that high-energy motion becomes bounded with plateau near $\hat B/(2c)$, decreasing approximately as $1/c$; the gate transition should occur near $E=E_\star$ with width about $4\tau$. The ungated divergence boundary should be shifted upward, and the measured plateau should agree with $\hat B/(2c)$ within approximately 20%.

Verification

Beats tuned baseline

Mechanism evidence: Confirmed

Practical benchmark: Beats tuned baseline

Stage 1 — Mechanism check agent confidence 9/10

Built the semi-passive energy-gated optimizer with explicit velocity, energy, logistic damping gate, toy dynamical verification, and a controlled digits MLP benchmark. The mechanism manifested: q(E) transitioned at E*=2 with the predicted 10–90% width, active unforced decay matched slope -2c within 0.08%, and forced active-regime plateaus matched G²/(2c²) essentially exactly. The optimizer did not improve this benchmark: at lr=0.3 momentum reached 98.22% accuracy versus 97.56% gated, although gating reduced maximum energy from 0.521 to 0.389.

Agent confidence: 9/10

Baseline: Momentum SGD: lr 0.03/0.1/0.3 accuracy 97.33%/95.78%/98.22%; max energy 0.0073/0.0549/0.5211.
Idea: Gated optimizer: lr 0.03/0.1/0.3 accuracy 97.11%/94.44%/97.56%; max energy 0.0068/0.0313/0.3891. Toy gate width relative error <1e-11; decay relative errors 0.01%-0.08%; forced plateau relative errors <=3.3e-5.

Limitations: The benchmark used sklearn digits rather than full MNIST, one fixed seed, three learning rates, and 300 updates. It did not test the proposed online exponential moving maximum estimate of B, Adam integration, larger networks, multiple seeds, or sustained pathological divergence regimes.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline Sep 1, 2026

Beats the tuned baseline by -0 metric points (p=0.0471, wins 6 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.0001
Effect (Δ): -0 (n/a; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.0471 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±22.7%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 0 configs

Limitations:

The baseline is Adam rather than momentum SGD because bench.train_model defines Adam as the canonical standard path. The experiment used 1200 training and 400 test samples, 15 epochs, and the built-in dynamics prediction task; it did not test full-scale datasets, momentum-SGD integration, the proposed online exponential moving maximum for B, or a direct quantitative continuous-time ultimate-bound calibration. The idea implementation uses bias-corrected Adam first-moment/preconditioned velocity with multiplicative gated damping, so it tests the implementation-plan Adam variant rather than raw M=I momentum.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Worked ✓

Agent confidence: 8/10

Built the semi-passive energy-gated optimizer as a local gated-Adam training loop on the matched dynamics track, using the shared rnn_small architecture and paired datasets. The idea achieved lower test MSE than the swept Adam baseline: 6.6809e-05 versus 8.2001e-05, delta=-1.5192e-05, with permutation p=0.04715, so this benchmark meets the harness criterion for a significant win. Trained-model behavior also showed nonzero active gating and positive damping energy terms, confirming the qualitative mechanism, although the original continuous-time B/(2c) plateau prediction was not directly tested as a quantitative bound.

Baseline: Adam sweep best lr=0.01, weight_decay=0.0; 8-seed mean dynamics test MSE 8.200101638067281e-05; per-seed [2.028812450589612e-04,1.6224317369051278e-04,4.15411705034785e-05,4.08045707445126e-05,5.192989920033142e-05,2.7118352591060102e-05,4.964483741787262e-05,7.984488183865324e-05]
Delta mean: -1.5192361161098233e-05
Idea: Gated Adam best cfg lr=0.01, c=10.0, E_star=0.01, tau=0.005; 8-seed mean dynamics test MSE 6.680865521957458e-05; per-seed [1.8218360492028296e-04,9.848197805695236e-05,3.606851532822475e-05,3.849564745905809e-05,5.274467548588291e-05,2.1851565179531462e-05,5.2063427574466914e-05,5.257982775219716e-05]
Idea wins: 6
P value: 0.04715

Benchmark result

Beats the tuned baseline by -0 metric points (p=0.0471, wins 6 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.0001
Effect (Δ): -0 (n/a; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.0471 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±22.7%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 0 configs

Limitations:

How to run: python3 bench_experiment.py

Artifacts

⬇ Download all as ZIP 6 files · code, reports and structured results

🐍 bench_experiment.py 6.3 KB View Raw JSON
📊 bench_report.json 6.9 KB View Raw JSON
🐍 experiment.py 5.5 KB View Raw JSON
📄 report.md 1.4 KB View
📄 report_bench_2026-09-01T234338.md 5.2 KB View
📊 results.json 4.4 KB View Raw JSON