Degree-Calibrated Stable Residual Flow
Implementation & benchmark of arXiv:2609.03120 — On a Gradation for Asymptotic Stability
Source paper: On a Gradation for Asymptotic Stability arXiv:2609.03120 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Construct a continuous-depth or recurrent residual block with a prescribed polynomial Lyapunov decay near its equilibrium. The architecture combines a fixed radial stabilizer with a learned component that is constrained to have zero radial projection, allowing slow algebraic transients and long memory while preventing asymptotic hidden-state growth.
Formulas
Mathematical statement
Let z(t) be a hidden state, z_star its equilibrium, r = ||z - z_star||^2 divided by 2, a greater than zero a stabilizing gain, and m greater than or equal to zero a chosen degree. Use the vector field dz/dt = -a r^m (z-z_star) + P_perp(z-z_star) h_phi(z,t), where P_perp(u) = I - uu^T/(||u||^2 + epsilon) approximately removes the radial component of the learned field. For the idealized field without projection error, dr/dt = -a r^(m+1). Thus r decays exponentially when m equals zero and as t^(-1/m) when m is positive; the state norm decays as t^(-1/(2m)). A sampled radial penalty can enforce the same inequality for the learned discretized vector field.
Implementation notes
Use this in a neural ODE, continuous-depth ResNet, or recurrent state-space block. Set z_star to zero initially and parameterize the hidden dynamics as f_phi(z,t,x) = -a r^m z + P_perp(z) h_phi(z,t,x). For an Euler block, update z_next = z + delta_t f_phi(z,t,x); for longer trajectories use an adaptive ODE solver. At sampled hidden states, compute r and the radial derivative z transpose f_phi(z), then add L_rad to the task loss. A hard alternative is to rescale any proposed update whose empirical radial derivative exceeds the allowed value. Compute the radial quantities exactly on minibatch states; estimate global validity by sampling states, inputs, and small perturbations because the penalty is not a global certificate. The paper supplies the degree-to-decay comparison; the tangent learned field and sampled penalty are the proposed neural adaptation. First test m equal to 0, 1, and 2 against an unconstrained residual MLP and a spectrally normalized residual block on a synthetic long-horizon sequence task and sequential MNIST. After the input is removed, measure hidden-state norms for 100 integration steps and plot r on log-log axes. The prediction is exponential decay for m equal to zero and a log-log slope near -1/m for r when m is positive, with no asymptotic growth provided the radial inequality holds. Increase delta_t to identify the numerical instability boundary and report whether the observed decay law persists below that boundary.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built and ran a deterministic toy implementation of the degree-calibrated residual flow with an exactly tangent learned component. The radial identity held to approximately 7e-13, and observed decay slopes were -1.002 for m=0, -0.965 for m=1, and -0.473 for m=2, matching the claimed exponential and algebraic laws. Against the unconstrained residual control, the calibrated flow stayed bounded under tested Euler steps through dt=1.0 while the control grew substantially; this supports the stability claim, though it is not a task-accuracy win.
- Agent confidence
- 8/10
- Baseline
- Unconstrained tanh residual dynamics: final maximum norm 5.64 at dt=0.01, 76.58 at dt=0.1, 407.74 at dt=0.5, and 830.43 at dt=1.0.
- Idea
- Degree-calibrated tangent flow: final maximum norm 0.706, 0.258, 0.159, and 0.198 at the same step sizes; maximum radial identity error 6.82e-13, with decay slopes -1.002 for m=0, -0.965 for m=1, and -0.473 for m=2.
Limitations: Only the toy hidden-state decay and stability phenomenon was tested; no sequence-learning accuracy, sequential MNIST, adaptive ODE solver, spectrally normalized baseline, FLOP comparison, or systematic Euler instability boundary was evaluated. With r=||z||^2/2, the supplied vector field needs a factor-of-two convention adjustment: the implementation uses -(a/2)r^m z so that dr/dt=-a r^(m+1), matching the stated decay claim.
How to run: python3 run_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2609.03120)
Significantly WORSE than the tuned baseline (+59.8%, p=0.0163, wins 1 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- residual_rnn_shared
- Paired seeds
- 8
- Baseline mean
- 0.0003
- Idea mean
- 0.0004
- Effect (Δ)
- 0.0002 (+59.8%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0163 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±28.9%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the structurally matched built-in dynamics track was tested; CIFAR, sequence, and tabular tracks were not run. The experiment used a small residual RNN and 18 epochs with dt fixed at 0.25; explicit Euler instability boundaries, adaptive ODE solvers, longer horizons, spectral-normalization baseline, FLOPs, and hidden-state decay after input removal were not evaluated. The m=1 configuration was much less accurate than m=0, and the baseline/idea architecture was a local residual-RNN implementation rather than the harness rnn_small GRU.
How to run: python3 bench_degree_flow.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented Degree-Calibrated Stable Residual Flow as a local residual recurrent architecture on the structurally matched actuated-pendulum dynamics track. The trained-model radial derivative matched -a*r^(m+1) with relative absolute error 1.43e-7 and correlation 0.9999999999999342, confirming the mechanism quantitatively. However, the idea's best result was significantly worse than the tuned baseline (MSE 0.00041295 vs 0.00025840, paired delta +0.00015455, permutation p=0.0163), so there was no neural-task accuracy win.
- Baseline
- Best unconstrained residual RNN, lr=0.006, dt=0.25: test MSE mean 0.0002584023, std 0.0000950261 across 8 seeds.
- Idea
- Best degree-calibrated residual RNN setting among tested settings, lr=0.006, dt=0.25, m=0: test MSE mean 0.0004129522, std 0.0001071165 across 8 seeds. Paired delta idea-baseline +0.0001545499; 1/8 idea wins; permutation p=0.0163; verdict idea worse (significant). Mechanism signature: predicted radial mean -4.6938066444, observed -4.6938066473, mean absolute error 5.94e-7, relative error 1.43e-7, correlation 0.9999999999999342, confirmed=true.
Benchmark result
Significantly WORSE than the tuned baseline (+59.8%, p=0.0163, wins 1 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- residual_rnn_shared
- Paired seeds
- 8
- Baseline mean
- 0.0003
- Idea mean
- 0.0004
- Effect (Δ)
- 0.0002 (+59.8%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0163 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±28.9%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the structurally matched built-in dynamics track was tested; CIFAR, sequence, and tabular tracks were not run. The experiment used a small residual RNN and 18 epochs with dt fixed at 0.25; explicit Euler instability boundaries, adaptive ODE solvers, longer horizons, spectral-normalization baseline, FLOPs, and hidden-state decay after input removal were not evaluated. The m=1 configuration was much less accurate than m=0, and the baseline/idea architecture was a local residual-RNN implementation rather than the harness rnn_small GRU.
How to run: python3 bench_degree_flow.py
Artifacts
- bench_degree_flow.py 5.3 KB View Raw
- bench_report.json 3.9 KB View Raw
- report.md 1.6 KB View
- report_bench_2026-09-04T155327.md 6.4 KB View
- results.json 2.2 KB View Raw
- run_experiment.py 3.8 KB View Raw