A Unified Discrete Gradient-SAV Framework for Structure-Preserving Integration

What the math gives to ML

The paper combines discrete-gradient chain rules with scalar auxiliary variables (SAV) so that an energy law is enforced exactly at the discrete time-step level while retaining inexpensive linear solves. This is transferable to neural optimization: reinterpret the training loss as an energy and use an auxiliary scalar to construct updates whose modified energy cannot increase for a fixed batch and positive-semidefinite preconditioner. The most practical first test is a deterministic or large-batch SAV optimizer with diagonal or block-diagonal curvature splitting, comparing loss monotonicity and convergence against AdamW and preconditioned gradient descent.