Finite-horizon Lyapunov regularization for neural updates

(1) Integrate at the optimizer-update or recurrent-state-update level. Start with AdamW training a small MLP or Transformer encoder. Maintain a detached snapshot q_k of the current optimization state: for the optimizer experiment use q_k=g_k, the minibatch gradient after global-norm clipping; for an SSM experiment use q_k=h_k-h^\star, where h^\star is a stop-gradient exponential moving average of hidden states. Do not backpropagate through the first V_k term or through the EMA mismatch estimate. (2) Pseudocode: choose M in {2,4,8}, alpha in {0.05,0.1}; at step k compute V_k=0.5*||q_k||^2; save q_k in a FIFO queue; after M updates obtain q_{k+M} and V_{k+M}=0.5*||q_{k+M}||^2; update eps_hat = beta*eps_hat + (1-beta)*abs(V_{k+M}-V_k+alpha*V_k); form r=V_{k+M}-V_k+alpha*V_k-eps_hat.detach(); add lambda*relu(r)^2 to the ordinary task loss or, for an optimizer test, add the penalty to a meta-loss over a short unrolled trajectory. (3) The paper supplies the finite-step decrease structure; estimate V directly from gradients or hidden states and estimate epsilon empirically from recent violations. Clip r and normalize V by its batch mean to avoid scale domination. Log the violation rate, V_{k+M}/V_k, gradient norm, and task loss. (4) First experiment: train a 2-layer 256-unit MLP on CIFAR-10 with SGD or AdamW, comparing the baseline against M=4 and M=8 at equal batch size, FLOPs, and learning-rate search budget. Run three random seeds. The primary success signal is a lower fraction of exploding or highly oscillatory loss windows and a smaller variance of gradient norms at equal final accuracy; a secondary target is improved validation accuracy under a 20-30% larger learning rate. For an SSM follow-up, compare hidden-state norm stability and perplexity on a small character-level language-model task.