# Average-contracting invariant fibre

- ID: 1240
- Canonical URL: https://synthcore.org/idea/1240/average-contracting-invariant-fibre
- API JSON: https://synthcore.org/api/idea/1240.json
- API Markdown: https://synthcore.org/api/idea/1240.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.21516](https://arxiv.org/abs/2607.21516)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, world-model, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, statistics, linear-algebra, functional-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace pointwise spectral-norm contraction in a recurrent or state-space model with an average logarithmic contraction certificate for an input-conditioned fibre update. Let a base state carry expressive, possibly noncontractive dynamics, while an auxiliary latent fibre contracts on average. This should preserve useful variability in the base while preventing long-horizon fibre explosion and making the fibre converge to an input-dependent invariant section.

## Mathematical statement

Represent the joint dynamics as a skew-product F(x,y) = (T(x), f_x(y)), where x is the base state, y is a fibre state, T is the base update, and f_x is the fibre update conditioned on x. If mu is an invariant distribution of T and L(x) bounds the fibre Lipschitz constant, the central assumption is chi = integral log L(x) dmu(x) < 0. The invariant section h satisfies h(T(x)) = f_x(h(x)). Two fibre trajectories driven by the same base sequence satisfy ||y_n - y'_n|| <= exp(sum from k=0 to n-1 of log L(x_k)) ||y_0 - y'_0||. Under ergodicity, the asymptotic perturbation rate is chi, so fibre perturbations decay approximately as exp(chi n). In a neural implementation, estimate L(x_t) with the spectral norm of the fibre Jacobian J_y f, and penalize the empirical average log gain when it exceeds a negative target margin gamma.

## Key formulas

- $$F(x,y)=(T(x),f_x(y)),\quad h(T(x))=f_x(h(x))$$
- $$chi=integral log L(x) dmu(x)<0,\quad L(x)=sup_{y != y'} ||f_x(y)-f_x(y')||/||y-y'||$$
- $$||y_n-y'_n|| <= exp(sum from k=0 to n-1 of log L(x_k)) ||y_0-y'_0|| approx exp(chi n)||y_0-y'_0||$$
- $$L_avg=[(1/(B T)) sum over b,t log(||J_y f(x_b,t,y_b,t)||_2+epsilon)+gamma]_+^2$$

## Implementation notes

1. Integration point: build a recurrent model with a base state x_t and an auxiliary fibre state y_t. Update x_(t+1) = T_theta(x_t,u_t) using an ordinary GRU, SSM, or transformer-derived state update. Update y_(t+1) = f_theta(x_t,y_t,u_t) using a gated MLP or linear state-space block, and decode from the concatenated state. Apply the stability penalty only to f, not to T, so the base retains expressive long-range dynamics. 2. Pseudocode: for each minibatch sequence, compute x_(t+1) = T(x_t,u_t), y_(t+1) = f(x_t,y_t,u_t); estimate g_t = log(||J_y f(x_t,y_t,u_t)||_2 + epsilon) using one or two power iterations, Hutchinson-style Jacobian-vector products, or an upper bound from layer spectral norms; accumulate chi_hat = mean(g_t); optimize task_loss + lambda times max(chi_hat + gamma, 0)^2. Periodically run two copies with identical inputs but different fibre initializations and record r_n = log(||y_n-y'_n||/||y_0-y'_0||)/n. 3. Taken from the paper: the skew-product decomposition, invariant-section equation, and negative average log-Lipschitz criterion. Estimated empirically: the base distribution, local Jacobian norms, finite-time exponent, and decay of correlations. 4. First cheap experiment: compare a vanilla GRU, a globally spectral-normalized GRU, and this two-state average-contracting GRU on sequential MNIST or permuted sequential MNIST, using 100- to 1000-step rollouts. Sweep fibre gain or penalty strength so chi_hat crosses zero. The predicted signature is a transition near chi_hat = 0: below zero, perturbation distances decay exponentially with slope approaching chi_hat; above zero, they grow or fail to forget initialization. With target margin gamma, the asymptotic measured slope should be near or below -gamma, and the empirical transition should agree with the sign boundary within approximately 20 percent.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a scalar skew-product fibre verification plus a tiny GRU base with an auxiliary affine fibre and average log-Jacobian penalty. The deterministic transition occurred at the predicted c=1 boundary, stochastic exponents matched E[log gain]=log(c)-sigma^2/2, and the margin sweep produced slopes within 1.3e-5 of -gamma. The neural regularizer changed fibre chi from +0.0499 to -0.1180, confirming contraction, but reduced task accuracy from 0.961 to 0.924, so no task-performance win was observed.

### Mechanism check

- Verdict: Built a scalar skew-product fibre verification plus a tiny GRU base with an auxiliary affine fibre and average log-Jacobian penalty. The deterministic transition occurred at the predicted c=1 boundary, stochastic exponents matched E[log gain]=log(c)-sigma^2/2, and the margin sweep produced slopes within 1.3e-5 of -gamma. The neural regularizer changed fibre chi from +0.0499 to -0.1180, confirming contraction, but reduced task accuracy from 0.961 to 0.924, so no task-performance win was observed.
- Confidence: 9/10
- Limitations: Only a scalar affine fibre and tiny synthetic sequence classification task were tested; no sequential MNIST, multidimensional Jacobian estimation, invariant-section reconstruction, FLOP/speed study, or generalization evaluation was performed. CUDA failed due to shared cuDNN host-allocation issues and the run fell back to CPU.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.228
- Paired wins: 5/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/258/bench_report.json)
- [experiment.py](https://synthcore.org/code/258/experiment.py)
- [report.md](https://synthcore.org/code/258/report.md)
- [report_bench_2026-08-30T234850.md](https://synthcore.org/code/258/report_bench_2026-08-30T234850.md)
- [results.json](https://synthcore.org/code/258/results.json)
- [stage2_bench.py](https://synthcore.org/code/258/stage2_bench.py)
- [stage2_stdout.json](https://synthcore.org/code/258/stage2_stdout.json)
- [Download all files as ZIP](https://synthcore.org/download/258)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.