✗ Failed on benchmark 2026

Pullback random-attractor monitor

Implementation & benchmark of arXiv:2608.29149 — Random attractors and almost-sure stability under discretization of a stochastic autoparametric system

Usefulness7/10
Difficulty4/10
Novelty8/10

Source paper: Random attractors and almost-sure stability under discretization of a stochastic autoparametric system arXiv:2608.29149 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the random-attractor construction as a training and inference diagnostic: initialize latent trajectories far in the past with different states but the same recent noise sequence, then measure whether they contract toward the same current set. This detects whether a stochastic recurrent model has a bounded, reproducible random attractor or instead exhibits discretization-induced divergence and spurious long-term modes.

Formulas

$$A_{n,h}(\omega)=\Phi_h^n(\theta_{-n}\omega,B),$$
$$\Phi_h(\omega,A_h(\omega))=A_h(\theta_1\omega),\qquad \operatorname{dist}\!\left(A_{n,h}(\omega),A_h(\omega)\right)\xrightarrow[n\to\infty]{}0,$$
$$D_{n,h}(\omega)=\max_{i,j}\left\|\Phi_h^n(\theta_{-n}\omega,z_i)-\Phi_h^n(\theta_{-n}\omega,z_j)\right\|\approx D_{0,h}e^{\lambda_hnh}\quad(\lambda_h<0).$$

Mathematical statement

Let $\Phi_h(\omega,z)$ be the stochastic neural one-step map, $\theta_{-n}\omega$ denote the noise sequence shifted $n$ steps into the past, and $B$ be a bounded set of initial latent states. The pullback image is $A_{n,h}(\omega)=\Phi_h^n(\theta_{-n}\omega,B)$. A random attractor is a family of compact sets $A_h(\omega)$ satisfying invariance $\Phi_h(\omega,A_h(\omega))=A_h(\theta_1\omega)$ and pullback attraction $\operatorname{dist}(A_{n,h}(\omega),A_h(\omega))\to0$. A negative top Lyapunov exponent predicts exponential contraction of the image diameter, $D_{n,h}\approx D_{0,h}e^{\lambda_hnh}$, until the attractor's intrinsic diameter is reached.

Implementation notes

Apply this monitor to a stochastic RNN, state-space world model, or diffusion latent transition model during validation and optionally as a regularizer. The same noise realization must be replayed across different initial states; otherwise ordinary noise variability is confused with pullback attraction. Sample a noise block $\omega_{-n:-1}$ and $K$ initial states $z_{-n}^{(i)}$ from a fixed bounded ball. Run every state through the identical sequence of stochastic updates and compute the pairwise diameter, or an RMS approximation, after every step. Repeat for several noise blocks and record $D_{n,h}$. If a stable attractor is desired, add $L_{\mathrm{attr}}=\sum_{r=r_0}^{R}[\log D_{r+1,h}-\log D_{r,h}-\widehat\lambda_hh]_+^2$, where $[x]_+=\max(x,0)$. Trajectories and diameters are directly computed; the attractor is approximated by the terminal point cloud $A_{R,h}$; the decay exponent is fitted by regressing $\log D_{n,h}$ against $nh$. First experiment: use a two-dimensional noisy RNN on a synthetic damped oscillator with 32 initial states and 128-step histories, comparing Euler-Maruyama at several step sizes with a semi-implicit update. The prediction is exponential pullback contraction whose fitted slope matches the independently measured $\widehat\lambda_h$. Near the stability boundary, the contraction time should grow approximately as $1/|\lambda_h|$; a coarse solver with a spurious positive exponent should show persistent or increasing diameter.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a pullback random-attractor monitor with shared-noise replay, Euler and semi-implicit oscillator updates, Lyapunov/spectral-radius checks, stability sweeps, and an independent-noise control. The mechanism manifested: Euler changed from contraction to divergence at the analytically predicted h≈0.20 boundary, measured pullback slopes closely matched log(rho)/h, and shared noise contracted to 0.0076 versus 0.5724 with independent noise. The 10%-diameter time prediction was accurate for stronger damping, though weaker cases were limited by finite horizon and transients.

Agent confidence
8/10
Baseline
Euler measured slopes at h=0.10, 0.18, 0.20, 0.22, 0.40 were -0.05058, -0.01002, -0.00009, +0.01004, +0.09618; it diverged above the predicted h=0.20 boundary. Independent-noise final diameter was 0.5724.
Idea
Semi-implicit remained contracting with slopes -0.09906, -0.09825, -0.09805, -0.09786, -0.09621 across the same step sizes. Shared-noise final diameter was 0.00759. For Euler damping 0.10, 0.20, 0.40, 0.80, measured pullback slopes were -0.02515, -0.07576, -0.17854, -0.39185 versus predicted -0.02503, -0.07528, -0.17655, -0.38221.

Limitations: Only a two-dimensional linear noisy oscillator was tested, not a learned stochastic RNN, SSM, world model, or training regularizer. The semi-implicit update is oscillator-specific; attractor-set geometry and long-run generalization were not evaluated.

How to run: python3 pullback_monitor.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.29149)

Ahead of the baseline (−6.4%) but not statistically significant (p=0.5999, wins 6 / 8 paired seeds).

Benchmark
Dynamics forecasting
Model
rnn_small
Paired seeds
8
Baseline mean
0.0021
Idea mean
0.002
Effect (Δ)
-0.0001 (−6.4%; negative = idea better)
Wins
6 / 8 paired seeds
p-value
0.5999 (permutation test, 20 000 shuffles)
Smallest detectable effect
±27.0%
Mechanism
Not confirmed ✗
Practical effect
Inconclusive
Baseline tuning
swept over 6 configs
Limitations:

Only the registered dynamics track and rnn_small model were tested. The regularizer sweep produced identical metrics, suggesting its effective gradient influence was inactive or ineffective; longer histories, alternative recurrent models, and a diagnostic-only inference study were not tested.

How to run: python3 bench_pullback.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 9/10

Built and evaluated a pullback-attractor regularizer on the structurally matched registered dynamics/rnn_small benchmark. The idea had lower mean MSE than baseline, but the paired permutation test was not significant (p=0.59995), and the trained-model contraction slope did not quantitatively match the prediction, so the idea did not establish a reliable win.

Baseline
Mean MSE 0.0020994337 (8 seeds), tuned lr=0.006 and weight_decay=0.0001.
Idea
Mean MSE 0.0019659589 (8 seeds), paired delta -0.0001334748, 6/8 wins, permutation p=0.59995.

Benchmark result

Ahead of the baseline (−6.4%) but not statistically significant (p=0.5999, wins 6 / 8 paired seeds).

Benchmark
Dynamics forecasting
Model
rnn_small
Paired seeds
8
Baseline mean
0.0021
Idea mean
0.002
Effect (Δ)
-0.0001 (−6.4%; negative = idea better)
Wins
6 / 8 paired seeds
p-value
0.5999 (permutation test, 20 000 shuffles)
Smallest detectable effect
±27.0%
Mechanism
Not confirmed ✗
Practical effect
Inconclusive
Baseline tuning
swept over 6 configs
Limitations:

Only the registered dynamics track and rnn_small model were tested. The regularizer sweep produced identical metrics, suggesting its effective gradient influence was inactive or ineffective; longer histories, alternative recurrent models, and a diagnostic-only inference study were not tested.

How to run: python3 bench_pullback.py

Artifacts

Implementation overview ⬇ Download all as ZIP 7 files · code, reports and structured results