On Energy Laws and Stability of First-Subdiagonal Pade Approximants for Linear Seminegative Problems
arXiv:2608.09239
2026
Dynamics
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper supplies a constructive stability mechanism for rational discretizations of dissipative but potentially non-normal linear dynamics: an exact discrete energy identity based on the operator-induced form [w,v]_L and an explicit Cholesky factorization of the Padé energy matrix. This can be transferred to neural dynamical blocks by replacing explicit residual updates with first-subdiagonal Padé maps whose step size is contractive for seminegative generators. The most practical experiment is a small learned linear or channel-mixing residual block, parameterized so its symmetric part is negative semidefinite, and compared against Euler, Cayley, and unconstrained residual updates at equal depth and compute.
Ideas from this paper
Unverified
2026
Replace an explicit residual layer x_{k+1}=x_k+hLx_k with a first-subdiagonal Padé rational layer. For the lowest nontrivial approximant, use R_{1,2}(z)=(1+z/3)/(1-2z/3+z^2/6), so x_{k+1}=R_{1,2}(hL)x_k; parameterize L to have a negative-semidefinite symmetric part, preventing exploding activations even for large learned step sizes.
Useful6/10
Difficulty6/10
Novelty6/10