Some properties of high-order nonstandard multistep multistage methods
arXiv:2607.08694
2026
Dynamics
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper develops nonstandard multistep-multistage and general linear ODE methods that replace the raw step size by a denominator function while retaining the formal order of standard schemes and preserving qualitative properties such as boundedness for arbitrary positive step sizes. The transferable asset is the combination of a block-state recurrence, a positive bounded effective step, and invariant-set assumptions that prevent numerical trajectories from leaving a safe region. A practical neural-network use is to build implicit or semi-implicit residual blocks whose internal dynamics use this nonstandard recurrence, then test whether large-depth or large-step networks remain stable without reducing the nominal step size.
Ideas from this paper
Unverified
2026
Replace the usual explicit residual update with a nonstandard general-linear block containing several internal feature stages. The effective step is a positive denominator function rather than the raw depth step, allowing the block to take large nominal steps while damping the update and preserving bounded activations. This is most promising for deep residual MLPs, neural ODE discretizations, and state-space sequence models where exploding hidden states limit usable depth.
Useful6/10
Difficulty6/10
Novelty6/10