△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Implementation & benchmark of arXiv:2607.17065 — A recursive subspace based method for errors-in-variables model identification of time-varying systems

Usefulness7/10

Difficulty6/10

Novelty7/10

Architecture Scalability Accuracy Stability Ssm World model Regularization Inference speedup Dynamical systems Linear algebra Statistics Spectral theory Numerical analysis

Source paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Formulas

$$r_k=\max\left\{r\in\{0,\ldots,r_{\max}\}:\lambda_{r,k}>\tau^{\mathrm{on}}_{r,k}\right\},\qquad S_k=U_k\Lambda_kU_k^\top,$$

$$\text{activate channel }i\text{ if }\lambda_{i,k}>\tau^{\mathrm{on}}_{i,k}\text{ for }M\text{ updates};\qquad \text{prune channel }i\text{ if }\lambda_{i,k}<\tau^{\mathrm{off}}_{i,k}\text{ for }M\text{ updates},$$

$$\tau^{\mathrm{on}}_{i,k}=c_{\mathrm{on}}\widehat\sigma^2_{i,k},\qquad \tau^{\mathrm{off}}_{i,k}=c_{\mathrm{off}}\widehat\sigma^2_{i,k},\qquad c_{\mathrm{on}}>c_{\mathrm{off}}>0.$$

Mathematical statement

Let $\lambda_{i,k}$ be the ordered eigenvalues of the recursively estimated noise-corrected covariance $S_k$, and let $\tau_{i,k}$ be a noise threshold derived from the estimated variance in channel $i$. Define active order $r_k=\max\{r:\lambda_{r,k}>\tau_{r,k}\}$, with hysteresis thresholds $\tau_i^{\mathrm{on}}>\tau_i^{\mathrm{off}}$ to prevent chattering. A latent SSM has active state $x_k\in\mathbb{R}^{r_k}$. When increasing order, append a state initialized from the new eigenvector and initialize its transition couplings near zero; when decreasing order, remove states whose eigenvalues remain below $\tau_i^{\mathrm{off}}$ for $M$ consecutive updates. The quantitative mechanism is a spectral crossing: architecture changes occur at $\lambda_i=\tau_i$, rather than at an arbitrary training epoch.

Implementation notes

1. Exact integration point: apply this controller to the state width of a diagonal-plus-low-rank SSM, gated RNN, or compact world model. Keep a maximum width $r_{\max}$ but maintain an active mask over state channels. The recursive covariance estimator runs on encoder outputs, while the neural transition is trained normally between structural updates. 2. Pseudocode: update $C$ and $R$ with the fixed-lag forgetting rule; eigendecompose $S=C-R$; for each ordered eigenvalue maintain an above/below counter; activate a channel after $M$ consecutive crossings of $\lambda_i>c_{\mathrm{on}}\widehat\sigma_i^2$; initialize its latent coordinate with $U_i^\top o$, set new transition weights to zero or small orthogonal values, and add it to the optimizer; prune after $M$ consecutive crossings of $\lambda_i<c_{\mathrm{off}}\widehat\sigma_i^2$, transferring its contribution to the decoder before removal. Freeze architecture changes for a cooldown interval to avoid rapid oscillation. 3. Computed from the paper's mechanism: recursive subspace spectrum, heteroskedastic noise correction, and fixed-window adaptation. Estimated empirically: constants $c_{\mathrm{on}}$, $c_{\mathrm{off}}$, persistence $M$, and maximum width. 4. First cheap experiment: train on a switched linear system whose true order changes from 2 to 5 and back to 2, with independent sensor-noise variance jumps. Compare fixed-width SSMs and magnitude-based neural pruning. The predicted signature is a sharp activation transition when $\lambda_{3,k}$ crosses $\tau^{\mathrm{on}}_{3,k}$ after the order increase, and pruning when it crosses $\tau^{\mathrm{off}}_{3,k}$ after the decrease. With hysteresis, active width should have substantially fewer toggles than a single-threshold controller, while multistep error should remain near the width-5 model during the high-order interval and near the width-2 model afterward.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a noise-corrected EWMA spectral rank controller with persistence and hysteresis, plus deterministic and stochastic toy sweeps. The mechanism manifested: the deterministic activation boundary occurred exactly at the predicted signal/noise ratio 2.0, adaptation delay was 6.5 steps on average versus 7 predicted, and hysteresis reduced toggles by 41.3%. In the switched-order reconstruction demo, active rank tracked 2→5→2 and adaptive MSE 0.0896 beat fixed rank-2 at 0.3072 and fixed rank-5 at 0.1232, although this is not yet evidence for a trained SSM win.

Agent confidence: 8/10

Baseline: Fixed rank-2 reconstruction MSE 0.3072; fixed rank-5 reconstruction MSE 0.1232; single-threshold controller 179 toggles.
Idea: Adaptive reconstruction MSE 0.0896; mean active ranks 1.98, 4.87, 2.07 across the 2→5→2 stream; hysteretic controller 105 toggles, a 41.3% reduction; deterministic spectral boundary at ratio 2.0, matching predicted c_on=2.0; noisy EWMA crossing delay mean 6.5 steps versus predicted 7.

Limitations: This is a compact covariance/controller MVP rather than a trained diagonal-plus-low-rank SSM or gated RNN. The toy uses independent latent channels and known isotropic noise; heteroskedastic online noise estimation, learned transitions, decoder transfer during pruning, optimizer-state management, speed, and realistic multistep forecasting were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.17065) Sep 2, 2026

Significantly WORSE than the tuned baseline (+55.2%, p=0.0303, wins 2 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0013
Idea mean: 0.002
Effect (Δ): 0.0007 (+55.2%; negative = idea better)
Wins: 2 / 8 paired seeds
p-value: 0.0303 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±43.4%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

This uses a fixed maximum-width GRU with masking rather than physically adding/removing optimizer parameters or a diagonal-plus-low-rank SSM. The noise floor is an empirical low-spectrum estimate; inference speed, optimizer-state transfer, decoder transfer during pruning, longer rollouts, and other tracks were not tested.

How to run: python3 stage2_rank_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented Noise-Triggered Latent Rank Adaptation as a covariance-controlled hysteretic active mask over a 64-channel GRU on the registered dynamics track. The correction math passed and trained models showed rank adaptation, but the adaptive system was significantly worse than the tuned fixed-width baseline: MSE 0.002004 versus 0.001291, paired delta +0.000713, permutation p=0.0303.

Baseline: Tuned fixed-width rnn_small, lr=0.006, 8 epochs: mean test MSE 0.001290830463403836 across 8 seeds.
Idea: Adaptive masked GRU, best of 3 settings: mean test MSE 0.0020039106311742216 across 8 seeds; paired delta +0.0007130801677703857; permutation p=0.0303.

Benchmark result