△ Mechanism confirmed, baseline not beaten 2026

Focus-Coefficient Switched Optimizer

Implementation & benchmark of arXiv:2607.17167 — Normal form method of center-focus problem in piecewise-smooth systems and algorithm design

Usefulness7/10

Difficulty6/10

Novelty8/10

Dynamics Stability Speedup Optimizer Training dynamics Scheduler Dynamical systems Bifurcations Control theory Optimization

Source paper: Normal form method of center-focus problem in piecewise-smooth systems and algorithm design arXiv:2607.17167 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Partition optimizer state space into regions and assign each region a different update rule, such as two learning rates, momentum values, or preconditioners. Fit the local radial normal form of the resulting piecewise-smooth training dynamics and switch to the branch whose first nonzero coefficient predicts contraction toward the stationary point.

Formulas

$$\dot z=F_\pm(z),\quad z=(x,y),\qquad \text{for }y\gtrless 0,$$

$$\frac{dr}{d\theta}=c_1r^3+c_2r^5+O(r^7),\qquad c_j\neq0\text{ for the first nonzero index }j,$$

$$c_j<0\ \Rightarrow\ r(\theta)\text{ decreases locally},\qquad c_j>0\ \Rightarrow\ r(\theta)\text{ increases locally},$$

$$s_{t+1}=\arg\min_{s\in\{+,-\}}\widehat c_{j,s},\qquad \theta_{t+1}=\theta_t+\eta_sU_s(\theta_t),$$

Mathematical statement

For a two-dimensional piecewise-smooth state z=(x,y) separated by a switching surface y=0, the paper transforms each local vector field into a normal form whose radial dynamics can be written as dr/dtheta=c_1 r^3+c_2 r^5+..., after the linear center term has been removed. The first nonzero coefficient c_j is algebraically equivalent to the corresponding Lyapunov constant: c_j<0 implies local inward spiraling and c_j>0 implies local outward spiraling, under the paper's orientation convention. For a neural optimizer, let z_t=(u_t,v_t) be a two-dimensional projection of the full parameter-and-momentum state, let s_t in {+,-} denote the active branch, and let F_+(z), F_-(z) be the expected projected updates under branch-specific hyperparameters. Estimate c_1 by fitting Delta r_t/r_t^3 near a candidate equilibrium; if c_1 is statistically indistinguishable from zero, fit c_2 using Delta r_t/r_t^5. Select the branch with the most negative reliable first nonzero coefficient. The mechanism doing the work is the normal-form reduction and sign test for the first nonzero focus coefficient.

Implementation notes

Use a small projected state for each training run rather than attempting to normal-form-reduce the full parameter vector. For Adam or SGD with momentum, define the full optimizer state q_t=(theta_t,m_t) and compute a two-dimensional projection z_t=(x_t,y_t) using two fixed random orthonormal directions, or use (loss_t minus running minimum, norm of m_t) as an initial diagnostic projection. Create two branches U_+ and U_- with different learning rates, momentum coefficients, or damping values. The switching surface is y=0 after centering and whitening the projected state. Every K steps, collect a short trajectory while r_t=norm(z_t-z_*) lies in a prescribed annulus [r_min,r_max]. Estimate radial drift by fitting Delta r_t=a_1r_t^3+a_2r_t^5 with robust regression separately for each branch; the paper contributes the normal-form interpretation, while coefficients are estimated empirically because the neural vector field is unavailable analytically. Ignore a coefficient if its bootstrap confidence interval contains zero, then use the first reliable coefficient. Pseudocode: run branch s for K steps; record r_t and r_{t+1}; fit c-hat_{1,s} and c-hat_{2,s}; choose the branch with the lowest reliable coefficient; impose hysteresis by requiring a coefficient margin delta before switching. First experiment: train a two-layer MLP on MNIST with SGD-momentum, comparing fixed SGD, cosine scheduling, and this two-branch controller. Sweep the base learning rate and measure local drift after deliberately perturbing parameters around a converged checkpoint. The quantitative prediction is a sign boundary: for each branch, measured mean radial drift should cross zero near the fitted normal-form threshold, and runs using a branch with c-hat_j<0 should show geometric local contraction while branches with c-hat_j>0 should show expansion. Falsify the method if the sign prediction fails in more than 20 percent of held-out perturbation trials or if it produces no improvement in estimated contraction rate.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable MVP implementing a radial normal-form switched dynamical system, RK4 integration, coefficient fitting, branch selection, and reproducible sweeps. The predicted c1 sign boundary was measured at -3.29e-6 versus the predicted 0, with maximum c1 fitting error 3.29e-6; the c2-leading sweep recovered c2 with maximum error 9.64e-4, and Delta-r/r^3 stayed within 0.7999996–0.7999999 across radii for true c1=0.8. Held-out branch selection chose the contracting branch in 90% of trials, meeting the stated 20% failure threshold, so the mathematical mechanism manifested in the toy system.

Agent confidence: 8/10

Baseline: Fixed outward branch final radius 0.20348; fixed inward branch final radius 0.16313
Idea: Coefficient-switched controller final radius 0.16649 from 0.18; held-out contracting-branch selection accuracy 90%; c1 boundary error 3.29e-6; c2 maximum estimation error 9.64e-4

Limitations: This verifies the focus-coefficient mechanism in a controlled two-dimensional toy vector field rather than a neural-network optimizer or MNIST MLP. It does not test real projected parameter/momentum trajectories, bootstrap confidence intervals, hysteresis, cosine scheduling, FLOPs, wall-clock speed, or GPU behavior; the fixed and switched comparison uses analytically specified branches rather than learned neural updates.

How to run: python3 focus_optimizer_mvp.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.17167) Sep 2, 2026

Ahead of the baseline (−3.8%) but not statistically significant (p=0.1747, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0002
Idea mean: 0.0002
Effect (Δ): -0 (−3.8%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.1747 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±5.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 6 configs

Limitations:

Only the registered dynamics track was tested, with reduced training and test sizes of 1200 and 400 examples and 12 epochs. The implementation uses a two-dimensional random parameter projection rather than a full normal-form reduction, and does not include bootstrap confidence intervals, explicit annulus filtering, formal hysteresis, FLOP accounting, or wall-clock comparisons.

How to run: python3 stage2_focus_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented and benchmarked the Focus-Coefficient Switched Optimizer on the registered dynamics track with the shared rnn_small architecture. The mechanism signature was confirmed from trained-model projected radial drift, but the task-metric improvement was not statistically significant, so the idea did not demonstrate a reliable win.

Baseline: Best SGD-momentum baseline: mean test MSE 0.00022353084659698652, std 0.00009061581347536935, lr=0.012, momentum=0.9
Idea: Best switched optimizer: mean test MSE 0.000214986320315802, std 0.00008761713194482366, paired delta=-0.000008544526281184517, 5/8 wins, permutation p=0.17475

Benchmark result