△ Mechanism confirmed, baseline not beaten 2026

Jacobian-Cocycle Growth Controller

Implementation & benchmark of arXiv:2607.09314 — Multi-type Galton-Watson processes in dynamical environments

Usefulness7/10

Difficulty5/10

Novelty7/10

Dynamics Stability Generalization Rnn Ssm Training dynamics Optimizer Dynamical systems Spectral theory Linear algebra Control theory

Source paper: Multi-type Galton-Watson processes in dynamical environments arXiv:2607.09314 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Treat the sequence of recurrent or state-space Jacobians along a trajectory as a noncommutative matrix cocycle, analogous to the time-dependent offspring mean matrices in the branching model. Estimate its finite-horizon growth exponent and use it to adapt spectral normalization or recurrent gain, targeting a slightly negative exponent for stable memory without uncontrolled exploding dynamics.

Formulas

$$||A||_infinity = max_(1 <= i <= d) sum_(j=1)^d |a_(i,j)|,qquad ||AB||_infinity <= ||A||_infinity ||B||_infinity$$

$$P_(t,n) = J_(t+n-1) ... J_t,\qquad gamma_t(n) = (1/n) log ||P_(t,n)||_infinity$$

$$gamma = limsup_(n to infinity) (1/n) log ||J_(n-1) ... J_0||_infinity,\qquad gamma < 0: contracting,\quad gamma = 0: critical,\quad gamma > 0: amplifying$$

$$L_cocycle = [max(0, gamma_hat_H - gamma_target)]^2,\qquad gamma_hat_H = (1/H) log ||J_(t+H-1) ... J_t||_2$$

Mathematical statement

Let the neural state update be h_(t+1) = F_theta(h_t, u_t), and let J_t = partial F_theta(h_t, u_t) / partial h_t be its d by d state Jacobian. The propagation over n steps is the ordered product P_(t,n) = J_(t+n-1) ... J_t. The branching-process analogue uses products of nonnegative mean reproduction matrices along the environment, with row-sum norm ||A||_infinity = max_i sum_j |a_(i,j)| and spectral radius rho(A). Define the finite-horizon cocycle growth exponent gamma_t(n) = n^(-1) log ||P_(t,n)||. When the asymptotic limit exists, gamma = lim_(n to infinity) gamma_t(n). A negative exponent implies exponential contraction, a positive exponent implies exponential amplification, and gamma = 0 is the critical boundary. Because products are noncommutative, constraining each rho(J_t) or ||J_t|| separately is only a sufficient condition and can miss cancellation or transient amplification. The controller estimates gamma over a short horizon and enforces a target gamma_target below zero using a differentiable penalty or multiplicative gain correction.

Implementation notes

1. Integration point: apply this to an RNN, gated state-space model, or continuous-depth residual block during training. For an RNN h_(t+1) = phi(W_h h_t + W_u u_t + b), compute J_t = Diag(phi'(W_h h_t + W_u u_t + b)) W_h. For a general cell, obtain Jacobian-vector products with automatic differentiation rather than explicitly materializing the Jacobian. 2. Pseudocode: sample a training sequence and unroll H states. Initialize a random unit vector v. At each time t compute v_new = J_t v, record r_t = log(norm(v_new)_2 + epsilon), and renormalize v = v_new / (norm(v_new)_2 + epsilon). Set gamma_hat_H = H^(-1) sum_t r_t. Add lambda times [max(0, gamma_hat_H - gamma_target)] squared to the task loss, or rescale the recurrent gain by g <- g exp(-eta_c (gamma_hat_H - gamma_target)) whenever the estimate is above target. Initially stop gradients through the controller if the optimization becomes unstable. 3. Computed versus estimated: Jacobian-vector products are computed exactly by autodiff, while the full matrix product and asymptotic exponent are estimated from one or several vectors over finite horizon H. Use multiple vectors with QR reorthogonalization if estimating more than the leading exponent. The paper supplies the cocycle growth mechanism and sign criterion; the target margin and controller gain are engineering choices. 4. First cheap experiment: compare a vanilla RNN, per-step spectral normalization, and this controller on sequential MNIST or the adding problem with hidden size 128 and horizons H equal to 20, 50, and 100. Sweep recurrent gain and plot task loss, gradient norm, tangent-perturbation norm, and gamma_hat_H. The falsifiable prediction is a sharp transition near gamma_hat_H = 0: positive-exponent runs should show exponential tangent growth and increasingly frequent exploding gradients, while negative-exponent runs should show decay approximately proportional to exp(n gamma_hat_H). Across gain sweeps, the observed divergence boundary should agree with the estimated zero crossing within 20 percent, and increasing H should stabilize the measured boundary rather than move it arbitrarily.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built and numerically verified the Jacobian-Cocycle Growth Controller. The toy predictions were confirmed: scalar growth crossed at gain 1.000, the noncommutative cocycle zero-growth gain was predicted as 1.05830 versus observed 1.05846, tangent growth matched exactly, and the controller converged to the analytically predicted gain 0.957586 with gamma=-0.10. In the secondary RNN test, the controller reduced the final estimated growth and gradient norm, but test loss was essentially unchanged (0.05273 baseline versus 0.05300 controller), so no task-level win was observed.

Agent confidence: 9/10

Baseline: Tiny tanh RNN with fixed recurrent gain 1.45: test MSE 0.052734, final estimated gamma 0.00117, final gradient norm 2.54.
Idea: Cocycle gain controller targeting gamma=-0.05: test MSE 0.052999, final gain 0.6731, estimated gamma -0.0302, final gradient norm 0.997. Toy mechanism: predicted versus observed zero gain 1.05830 vs 1.05846; tangent gamma 0.1256612 vs 0.1256612; controller predicted versus final gain 0.9575860940 vs 0.9575860940.

Limitations: The primary verification uses a small synthetic 4-dimensional matrix cocycle and finite horizon H=100, with exact products available; it does not establish asymptotic behavior for arbitrary cocycles. The RNN comparison is only one 32-unit, 40-step adding-like task and one seed, and compares fixed gain rather than a full modern spectral-normalization baseline. No sequential MNIST, multi-vector QR estimator, training-overhead measurement, or broad hyperparameter sweep was tested.

How to run: python3 cocycle_experiment.py && /home/maxwelhelp/main/bin/python3 rnn_mini_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.09314) Sep 2, 2026

No significant difference vs. the tuned baseline (0 metric points, p=1, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): 0 (n/a; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 0 configs

Limitations:

The benchmark used 14 epochs, 1000 training samples, and 300 test samples rather than the full default dataset sizes. Only one dynamics architecture and one short horizon were tested; no spectral-normalization baseline, multi-vector QR estimator, gradient-stability metric, longer training schedule, or broader target/eta sweep was included. The selected regime was already strongly contracting, so the controller's practical intervention was effectively inactive and cannot establish benefits near an exploding-dynamics boundary.

How to run: python3 stage2_cocycle_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Built the Jacobian-Cocycle Growth Controller on the structurally matched actuated-pendulum dynamics track using an explicit 64-unit GRU matching rnn_small. The trained-model tangent signature was confirmed with mean relative prediction error 2.60%, but the best idea configuration had exactly the same test MSE as the tuned baseline across all 8 paired seeds (delta_mean=0, p=1.0), yielding no measurable task-level effect. Therefore worked=false.

Baseline: Tuned rnn_small-compatible GRU, 9-config sweep over lr={0.001,0.003,0.005} and recurrent gain={1.1,1.4,1.7}; best full 8-seed mean test MSE 0.0001379394571, std 0.0000270793806, per-seed [0.0001238187,0.0001580492,0.0001153831,0.0001375052,0.0001933596,0.0001064980,0.0001161579,0.0001527439].
Idea: Cocycle controller, 3-config sweep over the same learning-rate union with targets {-0.03,-0.08,-0.15}, eta=0.25, initial gain=1.7; best full 8-seed mean test MSE 0.0001379394571, identical per-seed results. Paired delta_mean=0.0, idea_wins=0/8, permutation p-value=1.0.
Mechanism signature: On trained idea models over horizon 8, exp(H*gamma_hat) versus directly propagated tangent norms had mean relative error 0.02605, so the finite-horizon cocycle growth estimate was quantitatively confirmed (confirmed=true).

Benchmark result