✗ Failed on benchmark 2026

Average-contracting invariant fibre

Implementation & benchmark of arXiv:2607.21516 — Decay of Correlations for Partially Hyperbolic Skew-Products

Usefulness8/10

Difficulty5/10

Novelty7/10

Dynamics Stability Accuracy Generalization Rnn Ssm World model Training dynamics Regularization Dynamical systems Control theory Statistics Linear algebra

Source paper: Decay of Correlations for Partially Hyperbolic Skew-Products arXiv:2607.21516 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace pointwise spectral-norm contraction in a recurrent or state-space model with an average logarithmic contraction certificate for an input-conditioned fibre update. Let a base state carry expressive, possibly noncontractive dynamics, while an auxiliary latent fibre contracts on average. This should preserve useful variability in the base while preventing long-horizon fibre explosion and making the fibre converge to an input-dependent invariant section.

Formulas

$$F(x,y)=(T(x),f_x(y)),\quad h(T(x))=f_x(h(x))$$

$$chi=integral log L(x) dmu(x)<0,\quad L(x)=sup_{y != y'} ||f_x(y)-f_x(y')||/||y-y'||$$

$$||y_n-y'_n|| <= exp(sum from k=0 to n-1 of log L(x_k)) ||y_0-y'_0|| approx exp(chi n)||y_0-y'_0||$$

$$L_avg=[(1/(B T)) sum over b,t log(||J_y f(x_b,t,y_b,t)||_2+epsilon)+gamma]_+^2$$

Mathematical statement

Represent the joint dynamics as a skew-product F(x,y) = (T(x), f_x(y)), where x is the base state, y is a fibre state, T is the base update, and f_x is the fibre update conditioned on x. If mu is an invariant distribution of T and L(x) bounds the fibre Lipschitz constant, the central assumption is chi = integral log L(x) dmu(x) < 0. The invariant section h satisfies h(T(x)) = f_x(h(x)). Two fibre trajectories driven by the same base sequence satisfy ||y_n - y'_n|| <= exp(sum from k=0 to n-1 of log L(x_k)) ||y_0 - y'_0||. Under ergodicity, the asymptotic perturbation rate is chi, so fibre perturbations decay approximately as exp(chi n). In a neural implementation, estimate L(x_t) with the spectral norm of the fibre Jacobian J_y f, and penalize the empirical average log gain when it exceeds a negative target margin gamma.

Implementation notes

1. Integration point: build a recurrent model with a base state x_t and an auxiliary fibre state y_t. Update x_(t+1) = T_theta(x_t,u_t) using an ordinary GRU, SSM, or transformer-derived state update. Update y_(t+1) = f_theta(x_t,y_t,u_t) using a gated MLP or linear state-space block, and decode from the concatenated state. Apply the stability penalty only to f, not to T, so the base retains expressive long-range dynamics. 2. Pseudocode: for each minibatch sequence, compute x_(t+1) = T(x_t,u_t), y_(t+1) = f(x_t,y_t,u_t); estimate g_t = log(||J_y f(x_t,y_t,u_t)||_2 + epsilon) using one or two power iterations, Hutchinson-style Jacobian-vector products, or an upper bound from layer spectral norms; accumulate chi_hat = mean(g_t); optimize task_loss + lambda times max(chi_hat + gamma, 0)^2. Periodically run two copies with identical inputs but different fibre initializations and record r_n = log(||y_n-y'_n||/||y_0-y'_0||)/n. 3. Taken from the paper: the skew-product decomposition, invariant-section equation, and negative average log-Lipschitz criterion. Estimated empirically: the base distribution, local Jacobian norms, finite-time exponent, and decay of correlations. 4. First cheap experiment: compare a vanilla GRU, a globally spectral-normalized GRU, and this two-state average-contracting GRU on sequential MNIST or permuted sequential MNIST, using 100- to 1000-step rollouts. Sweep fibre gain or penalty strength so chi_hat crosses zero. The predicted signature is a transition near chi_hat = 0: below zero, perturbation distances decay exponentially with slope approaching chi_hat; above zero, they grow or fail to forget initialization. With target margin gamma, the asymptotic measured slope should be near or below -gamma, and the empirical transition should agree with the sign boundary within approximately 20 percent.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a scalar skew-product fibre verification plus a tiny GRU base with an auxiliary affine fibre and average log-Jacobian penalty. The deterministic transition occurred at the predicted c=1 boundary, stochastic exponents matched E[log gain]=log(c)-sigma^2/2, and the margin sweep produced slopes within 1.3e-5 of -gamma. The neural regularizer changed fibre chi from +0.0499 to -0.1180, confirming contraction, but reduced task accuracy from 0.961 to 0.924, so no task-performance win was observed.

Agent confidence: 9/10

Baseline: GRU task accuracy 0.9609, loss 0.1124, fibre chi +0.0499; deterministic transition at c=1.0; stochastic predicted boundary c=1.1972
Idea: Task accuracy 0.9238, loss 0.1651, fibre chi -0.1180 and perturbation rate -0.1180; margin slopes for gamma={0.05,0.10,0.20,0.40} differed from prediction by at most 1.3e-5; stochastic exponents closely matched predictions

Limitations: Only a scalar affine fibre and tiny synthetic sequence classification task were tested; no sequential MNIST, multidimensional Jacobian estimation, invariant-section reconstruction, FLOP/speed study, or generalization evaluation was performed. CUDA failed due to shared cuDNN host-allocation issues and the run fell back to CPU.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.21516) Sep 2, 2026

Ahead of the baseline (−40.0%) but not statistically significant (p=0.228, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0002
Idea mean: 0.0001
Effect (Δ): -0.0001 (−40.0%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.228 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested. The experiment used 1200 training and 400 test examples, 12 epochs, and a small three-value learning-rate baseline sweep; no larger datasets, longer training, alternative recurrent architectures, or additional dynamics benchmarks were tested. The local Jacobian estimate used the exact scalar fibre derivative for this implementation, while the signature additionally used finite perturbation rollouts.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented the average-contracting invariant-fibre intervention on the structurally matched dynamics track using paired end-to-end GRU systems and the required baseline/idea sweeps. The idea reduced test MSE from 0.00015379 to 0.00009226, but the paired permutation test was not significant (p=0.228), so there is no validated benchmark win. The corrected trained-model perturbation signature was also not quantitatively confirmed because the observed finite-time rate differed from the local-log-Jacobian prediction by 0.4005.

Baseline: dynamics/rnn_small; best lr=0.008; test MSE 0.0001537921 ± 0.0001087466 over 8 seeds
Idea: average fibre penalty lambda=0.5, lr=0.008; test MSE 0.0000922559 ± 0.0000460421 over 8 seeds; paired delta=-0.0000615362; p=0.228; 5/8 wins
Mechanism signature: baseline local chi=-0.7256952; idea local chi=-0.3251547; observed-minus-predicted perturbation rate=0.4005406; confirmed=false

Benchmark result