# Polynomial-Lyapunov Training Controller

- ID: 3126
- Canonical URL: https://synthcore.org/idea/3126/polynomial-lyapunov-training-controller
- API JSON: https://synthcore.org/api/idea/3126.json
- API Markdown: https://synthcore.org/api/idea/3126.md
- Verification status: unverified
- Source: [arXiv:2609.03120](https://arxiv.org/abs/2609.03120)
- Category: dynamics
- Solves: stability, speedup
- ML areas: optimizer, training-dynamics, scheduler
- Math tags: dynamical-systems, control-theory, optimization
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Treat the optimization error as a Lyapunov-like state and adapt the learning rate so that its measured decrease follows a chosen stability degree. Instead of requiring exponential decrease, the controller targets dE/dt approximately equal to -c E^(1+m), which is appropriate near flat minima or marginally stable training regimes where exponential contraction may be impossible.

## Mathematical statement

Let E(t) be a nonnegative training error, c greater than zero a dissipation constant, and m greater than or equal to zero the stability degree. If dE/dt is at most -c E^(1+m), comparison theory gives exponential decay E(t) at most E(0) exp(-ct) for m equal to zero, and algebraic decay E(t) at most [E(0)^(-m) + mct]^(-1/m) for m greater than zero. In a discrete optimizer, estimate c at step k by c_hat_k = (E_k - E_(k+1)) divided by delta_t E_k^(1+m). Adjust the step size toward a target c_star while rejecting steps that violate monotonic decrease. The measurable asymptotic signature is a log-log slope of -1/m for m greater than zero.

## Key formulas

- $$dE/dt <= -c E^(1+m),   c > 0,   m >= 0$$
- $$E(t) <= E(0) exp(-ct)  if m = 0;   E(t) <= [E(0)^(-m) + mct]^(-1/m)  if m > 0$$
- $$c_hat_k = (E_k - E_(k+1))/(delta_t E_k^(1+m))$$
- $$eta_k <- eta_k times clip((c_hat_k/c_star)^alpha, eta_min/eta_k, eta_max/eta_k)$$

## Implementation notes

Integrate this controller into SGD or Adam after computing a minibatch gradient but before permanently applying the update. Maintain a nonnegative monitored error E_k: on supervised learning use the current loss minus a running lower bound, clipped below by epsilon_E. Pseudocode: compute gradient g_k; propose theta_trial = theta_k - eta_k g_k; evaluate a small fixed monitoring minibatch and obtain E_trial; compute c_hat = (E_k - E_trial)/(E_k^(1+m) delta_t); reject the proposal and halve eta if E_trial exceeds E_k; otherwise accept it and update eta using the displayed multiplicative controller toward c_star. Smooth c_hat with an exponential moving average to reduce minibatch noise. The Lyapunov comparison law and degree interpretation come from the paper; c_star, the loss floor, smoothing coefficient, and rejection thresholds are empirical. First run a two-layer MLP on MNIST and a small ResNet on CIFAR-10, comparing SGD, Adam, and controllers with m equal to 0, 1, and 2. Plot loss on log-linear and log-log axes. The falsifiable prediction is exponential decay for m equal to zero and a final log-log slope near -1/m for m greater than zero; estimate the slope over the final third of training and require agreement within 20 percent. Also test learning rates around the acceptance boundary: the controller should reject steps when c_hat becomes nonpositive, and the measured transition from stable to divergent updates should be reproducible across random seeds.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.