# Periodic Lyapunov Guard for Cyclic Training

- ID: 2597
- Canonical URL: https://synthcore.org/idea/2597/periodic-lyapunov-guard-for-cyclic-training
- API JSON: https://synthcore.org/api/idea/2597.json
- API Markdown: https://synthcore.org/api/idea/2597.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.19975](https://arxiv.org/abs/2608.19975)
- Category: dynamics
- Solves: stability, speedup
- ML areas: optimizer, training-dynamics, scheduler, rnn
- Math tags: dynamical-systems, control-theory, linear-algebra, spectral-theory, optimization
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Model one period of a cyclic optimizer or periodically modulated recurrent network as a discrete-time linear time-periodic system obtained by linearizing the update around its current trajectory. Estimate a periodic Lyapunov matrix sequence and scale the next learning-rate or modulation amplitude so that every phase contracts according to a certified energy decrease. This should prevent delayed divergence caused by resonance with the schedule, even when individual phase Jacobians are temporarily expansive.

## Mathematical statement

Let the neural state be $z_k$, comprising parameters, hidden state, or both, and let one schedule period have $M$ phases. Linearization at phase $k$ gives $\delta z_{k+1}=J_k\delta z_k$, where $J_k=\partial F_k/\partial z$ and $J_{k+M}=J_k$. A periodic quadratic Lyapunov function is $V_k(\delta z)=\delta z^\top P_k\delta z$, with symmetric positive-definite $P_k=P_k^\top\succ0$ and $P_{k+M}=P_k$. The paper's LTP stability mechanism becomes the discrete periodic LMI $J_k^\top P_{k+1}J_k-P_k\preceq-Q_k$, where $Q_k\succ0$. It implies $V_{k+1}-V_k\le-\delta z_k^\top Q_k\delta z_k$, hence local exponential stability. Over one period, the monodromy matrix is $\Phi=J_{M-1}\cdots J_0$; feasibility implies $\rho(\Phi)<1$, where $\rho$ is spectral radius. For a scalar safety margin $\gamma$, enforce $J_k^\top P_{k+1}J_k\preceq\gamma^2P_k$; then perturbations contract by at least $\gamma$ per phase in the certified metric and by at most $\gamma^M$ per period.

## Key formulas

- $$\delta z_{k+1}=J_k\delta z_k,\qquad J_k=\left.\frac{\partial F_k(z)}{\partial z}\right|_{z=\bar z_k},\qquad J_{k+M}=J_k.$$
- $$J_k^{\top}P_{k+1}J_k-P_k\preceq-Q_k,\qquad P_k=P_k^{\top}\succ0,\quad Q_k=Q_k^{\top}\succ0.$$
- $$\Phi=J_{M-1}J_{M-2}\cdots J_0,\qquad \rho(\Phi)<1,$$
- $$J_k^{\top}P_{k+1}J_k\preceq\gamma^2P_k\ \forall k\quad\Longrightarrow\quad V_{k+M}\le\gamma^{2M}V_k.$$

## Implementation notes

1. Integration point: use this as a controller around a cyclic learning-rate optimizer, momentum optimizer, or recurrent inference loop. Define the phase-specific update $z_{k+1}=F_k(z_k)$; for ordinary training, $z_k$ can concatenate flattened parameters and optimizer momentum, while for an RNN it can concatenate hidden state and a local parameter subset. At the end of every schedule period, freeze a minibatch and estimate the Jacobians $J_k$ with forward-mode JVPs or finite differences on a low-dimensional random subspace. 2. Pseudocode: collect $J_0,\ldots,J_{M-1}$; solve an SDP for $P_0,\ldots,P_{M-1}\succ\epsilon I$ minimizing $\gamma$ subject to $J_k^\top P_{k+1}J_k\preceq\gamma^2P_k$ and $P_M=P_0$; if $\gamma>\gamma_{\max}$, multiply the learning rate and schedule amplitude by $\gamma_{\max}/\gamma$ and retry; otherwise perform the next period. For large models, use diagonal or block-diagonal $P_k$ and Jacobian sketches instead of a full SDP. 3. Computed from the paper's mechanism: the periodic Lyapunov inequalities, phase-dependent energy, and monodromy stability criterion. Estimated empirically: the Jacobians, local trajectory, and SDP margin. 4. First cheap experiment: train a two-layer MLP on MNIST with cosine learning rate of period $M=20$, comparing SGD, gradient clipping, and the Lyapunov guard using a diagonal $P_k$ on a 100-dimensional parameter projection. Sweep the base learning rate and measure the largest stable rate. Prediction: divergence begins near the measured boundary $\rho(\Phi)=1$; below it, squared perturbation energy should decay approximately as $\gamma^{2t}$, and the guarded run should maintain $\gamma<0.95$ while unguarded runs become unstable when $\rho(\Phi)>1$.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a deterministic two-phase periodic Lyapunov guard with diagonal periodic certificates and adaptive scalar-rate scaling. The mechanism manifested: the predicted rho(Phi)=1 boundary was alpha=0.833333 versus a sweep observation of 0.835, predicted and certified gamma matched to floating-point precision, and period-energy ratios matched the certified gamma^(2M) bounds. At requested alpha=1.0, the unguarded system diverged to norm 2.32e7 while the guard selected alpha=0.809835 and reduced the final norm to 0.0461. This validates the toy mechanism, but not a neural-training win.

### Mechanism check

- Verdict: Built a deterministic two-phase periodic Lyapunov guard with diagonal periodic certificates and adaptive scalar-rate scaling. The mechanism manifested: the predicted rho(Phi)=1 boundary was alpha=0.833333 versus a sweep observation of 0.835, predicted and certified gamma matched to floating-point precision, and period-energy ratios matched the certified gamma^(2M) bounds. At requested alpha=1.0, the unguarded system diverged to norm 2.32e7 while the guard selected alpha=0.809835 and reduced the final norm to 0.0461. This validates the toy mechanism, but not a neural-training win.
- Confidence: 9/10
- Limitations: Only a 2D diagonal linear time-periodic toy system was tested. No MNIST/MLP training, stochastic minibatches, momentum, Jacobian estimation error, non-normal dynamics, SDP solver, runtime, FLOPs, or GPU experiment was evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.1569
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [REPORT.md](https://synthcore.org/code/905/REPORT.md)
- [bench_report.json](https://synthcore.org/code/905/bench_report.json)
- [periodic_guard_experiment.py](https://synthcore.org/code/905/periodic_guard_experiment.py)
- [report.md](https://synthcore.org/code/905/report.md)
- [report_bench_2026-09-01T143318.md](https://synthcore.org/code/905/report_bench_2026-09-01T143318.md)
- [results.json](https://synthcore.org/code/905/results.json)
- [stage2_bench.py](https://synthcore.org/code/905/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/905)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.