Explicit stabilized implementation of singly diagonally implicit Runge-Kutta methods

What the math gives to ML

The paper gives a constructive way to obtain the stability benefits of singly diagonally implicit Runge–Kutta (SDIRK) methods while replacing each nonlinear implicit stage solve by an explicitly stabilized auxiliary integration. The transferable asset is the separation between the outer high-order SDIRK discretization and an inner polynomially stabilized fixed-point solver, which is relevant to stiff neural ODEs, continuous-depth residual networks, and equilibrium layers. A practical neural adaptation is to use the supplied stiffly accurate SDIRK4 stages, but solve each diagonal stage with Chebyshev-accelerated residual iterations using Jacobian-vector products rather than Newton factorizations. The main hypothesis is that this permits substantially larger time steps than explicit RK4 at comparable network evaluations, provided the learned dynamics have predominantly dissipative Jacobian spectra.