Degree-Calibrated Stable Residual Flow
Implementation & benchmark of arXiv:2609.03120 — On a Gradation for Asymptotic Stability
Source paper: On a Gradation for Asymptotic Stability arXiv:2609.03120 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Construct a continuous-depth or recurrent residual block with a prescribed polynomial Lyapunov decay near its equilibrium. The architecture combines a fixed radial stabilizer with a learned component that is constrained to have zero radial projection, allowing slow algebraic transients and long memory while preventing asymptotic hidden-state growth.
Formulas
Mathematical statement
Let z(t) be a hidden state, z_star its equilibrium, r = ||z - z_star||^2 divided by 2, a greater than zero a stabilizing gain, and m greater than or equal to zero a chosen degree. Use the vector field dz/dt = -a r^m (z-z_star) + P_perp(z-z_star) h_phi(z,t), where P_perp(u) = I - uu^T/(||u||^2 + epsilon) approximately removes the radial component of the learned field. For the idealized field without projection error, dr/dt = -a r^(m+1). Thus r decays exponentially when m equals zero and as t^(-1/m) when m is positive; the state norm decays as t^(-1/(2m)). A sampled radial penalty can enforce the same inequality for the learned discretized vector field.
Implementation notes
Use this in a neural ODE, continuous-depth ResNet, or recurrent state-space block. Set z_star to zero initially and parameterize the hidden dynamics as f_phi(z,t,x) = -a r^m z + P_perp(z) h_phi(z,t,x). For an Euler block, update z_next = z + delta_t f_phi(z,t,x); for longer trajectories use an adaptive ODE solver. At sampled hidden states, compute r and the radial derivative z transpose f_phi(z), then add L_rad to the task loss. A hard alternative is to rescale any proposed update whose empirical radial derivative exceeds the allowed value. Compute the radial quantities exactly on minibatch states; estimate global validity by sampling states, inputs, and small perturbations because the penalty is not a global certificate. The paper supplies the degree-to-decay comparison; the tangent learned field and sampled penalty are the proposed neural adaptation. First test m equal to 0, 1, and 2 against an unconstrained residual MLP and a spectrally normalized residual block on a synthetic long-horizon sequence task and sequential MNIST. After the input is removed, measure hidden-state norms for 100 integration steps and plot r on log-log axes. The prediction is exponential decay for m equal to zero and a log-log slope near -1/m for r when m is positive, with no asymptotic growth provided the radial inequality holds. Increase delta_t to identify the numerical instability boundary and report whether the observed decay law persists below that boundary.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.