Finite-Horizon Lyapunov Risk Monitor
Implementation & benchmark of arXiv:2607.26963 — Finite-Time Chaos Diagnostics and Noise-Induced Basin Merging in a Two-Dimensional Map
Source paper: Finite-Time Chaos Diagnostics and Noise-Induced Basin Merging in a Two-Dimensional Map arXiv:2607.26963 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Treat the hidden-state evolution of an RNN or state-space model as a randomly perturbed map and estimate the distribution of finite-time expansion rates rather than only the spectral radius of an average Jacobian. Penalize high-probability positive FTLEs, allowing the model to remain expressive while controlling rare finite-horizon explosions.
Formulas
Mathematical statement
For a stochastic map x_{t+1}=f_{theta_t}(x_t) with independently sampled parameters theta_t, define lambda_T=(1/T) log ||J_{T-1}...J_0||, where J_t=Df_{theta_t}(x_t) is the hidden-state Jacobian and ||.|| is the spectral norm. Let mu_{T,sigma}=E[lambda_T] and v_{T,sigma}=Var(lambda_T) under perturbation amplitude sigma. The paper's finite-time Gaussian mechanism predicts (lambda_T-mu_{T,sigma})/sqrt(v_{T,sigma}) is approximately standard normal. Therefore the probability of finite-horizon expansion is p_+=P(lambda_T>0) approximately equal to 1-Phi(-mu_{T,sigma}/sqrt(v_{T,sigma})). A one-sided upper-confidence penalty controls this probability directly.
Implementation notes
Integration point: apply this to an RNN, gated state-space model, or discretized neural ODE with hidden update h_{t+1}=F_theta(h_t,u_t). During training, inject independently resampled parameter noise or input noise at every unrolled step and measure the hidden-state Jacobian. Pseudocode: for each of K perturbation realizations, initialize a random unit vector q and set s=0; for t=0,...,T-1 compute q=J_t q using a Jacobian-vector product, add log(||q||+epsilon) to s, and renormalize q; store lambda=s/T. Estimate mu_hat and s_hat from the K samples and add the displayed upper-confidence penalty to the task loss. The paper supplies the product-Jacobian FTLE definition and Gaussian standardization. The mean, variance, Jacobian-vector products, and Gaussianity are estimated empirically; no full Jacobian is needed. First experiment: train tanh RNNs on sequential MNIST or the adding problem, comparing no penalty, a mean-FTLE penalty, and the upper-confidence penalty. Sweep T and noise amplitude sigma. The prediction is that standardized FTLE histograms become approximately normal and that the observed positive-FTLE fraction agrees with p_+ within about 10 percentage points. A divergence boundary should occur near mu_hat+z_{0.95}s_hat=0; models below this boundary should exhibit fewer long-horizon hidden-state explosions.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Inconclusive
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a differentiable finite-horizon FTLE monitor for a noisy tanh RNN, including mean and one-sided UCB risk variants, plus an exact scalar stochastic-map verification. In the toy sweep, observed standard deviations matched sigma/sqrt(T) within about 0.1%, tail probabilities matched the Gaussian prediction closely, and the UCB crossed zero at sigma=0.195 versus the predicted 0.1945. The mechanism is therefore numerically manifested, although the RNN comparison is not evidence of a practical task-performance win because the unregularized baseline was already highly contractive.
- Agent confidence
- 9/10
- Baseline
- RNN MSE 0.1655; FTLE mean -0.6946; positive fraction 0.000; UCB -0.5491
- Idea
- UCB RNN MSE 0.1679; FTLE mean -0.3960; positive fraction 0.000; UCB -0.2678. Mean-only variant: MSE 0.1722, FTLE mean -0.0340, positive fraction 0.359, UCB 0.1116. Toy: sd scaling and p+ predictions matched; UCB boundary observed 0.195 vs predicted 0.1945.
Limitations: The scalar verification is an analytically Gaussian construction, so it validates the formulas but not Gaussianity for nonlinear RNN dynamics. The RNN study used a tiny adding-style task, one noise level, short training, K=4 monitor samples, and did not test sequential MNIST, FLOPs, long-horizon generalization, or multiple random seeds.
How to run: python3 experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.26963)
Ahead of the baseline (−7.2%) but not statistically significant (p=0.1703, wins 7 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0016
- Idea mean
- 0.0015
- Effect (Δ)
- -0.0001 (−7.2%; negative = idea better)
- Wins
- 7 / 8 paired seeds
- p-value
- 0.1703 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±11.5%
- Mechanism
- Confirmed ✓
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 2 configs
The official README was present on disk but inaccessible through the read tool; its contract and APIs were inspected directly via Python. The implementation used a local GRUCell equivalent to expose hidden states for the intervention rather than bench.make_model's opaque GRU wrapper. The finite-difference FTLE monitor is a directional Jacobian proxy rather than an exact spectral norm, and the three rho settings produced identical reported metrics, indicating the regularization term was effectively inactive or numerically negligible in this short run. The protocol used a reduced 2-value baseline learning-rate sweep and 4 training epochs to fit the runtime budget.
How to run: python3 bench_run.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented the Finite-Horizon Lyapunov Risk Monitor on the registered dynamics track using matched recurrent systems and evaluated it with the official bench reporting protocol. The idea achieved lower mean MSE (0.0014701 vs 0.0015849; paired delta -0.0001148) and won 7/8 pairs, but permutation p=0.17025, so this is not a significant win. The trained-model mechanism signature was confirmed: predicted positive-FTLE fraction 6.10e-28 versus observed 0.0.
- Baseline
- Tuned baseline rnn_small: mean test MSE 0.0015848712, std 0.0004409408, best lr 0.003
- Idea
- Best reported UCB monitor: mean test MSE 0.0014701014, std 0.0005500, paired delta -0.0001147698, 7/8 wins, permutation p=0.17025
- Mechanism signature
- Gaussian FTLE prediction: predicted positive fraction 6.103742223005928e-28, observed 0.0, absolute error 6.103742223005928e-28, confirmed=true
Benchmark result
Ahead of the baseline (−7.2%) but not statistically significant (p=0.1703, wins 7 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0016
- Idea mean
- 0.0015
- Effect (Δ)
- -0.0001 (−7.2%; negative = idea better)
- Wins
- 7 / 8 paired seeds
- p-value
- 0.1703 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±11.5%
- Mechanism
- Confirmed ✓
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 2 configs
The official README was present on disk but inaccessible through the read tool; its contract and APIs were inspected directly via Python. The implementation used a local GRUCell equivalent to expose hidden states for the intervention rather than bench.make_model's opaque GRU wrapper. The finite-difference FTLE monitor is a directional Jacobian proxy rather than an exact spectral norm, and the three rho settings produced identical reported metrics, indicating the regularization term was effectively inactive or numerically negligible in this short run. The protocol used a reduced 2-value baseline learning-rate sweep and 4 training epochs to fit the runtime budget.
How to run: python3 bench_run.py
Artifacts
- bench_report.json 3.9 KB View Raw
- bench_run.py 5.1 KB View Raw
- experiment.py 5.5 KB View Raw
- flrm_dynamics.py 4.8 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-08-31T183940.md 4.2 KB View
- results.json 4.3 KB View Raw