# Subspace-Restarted State-Space Dynamics

- ID: 825
- Canonical URL: https://synthcore.org/idea/825/subspace-restarted-state-space-dynamics
- API JSON: https://synthcore.org/api/idea/825.json
- API Markdown: https://synthcore.org/api/idea/825.md
- Verification status: unverified
- Source: [arXiv:2607.12727](https://arxiv.org/abs/2607.12727)
- Category: dynamics
- Solves: stability, accuracy, memory
- ML areas: rnn, ssm, world-model, training-dynamics
- Math tags: dynamical-systems, stochastic-processes, statistical-mechanics, control-theory
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Split a recurrent or state-space model into a persistent slow state and a fast internal state. Every r recurrent steps, preserve the slow state but reset or contract the fast state toward a learned reference, reproducing selective restart rather than a destructive global reset. The expected benefit is suppression of long-range oscillatory and error correlations while retaining trajectory-level information.

## Mathematical statement

The source process evolves as |psi(n)> = W^n|psi(0)> and has spatial probability P(x,n) = sum_s |alpha_s(x,n|0)|^2. A restart every r steps resets internal degrees of freedom while preserving the spatial distribution. The neural adaptation uses z_t = (h_t,c_t), where h_t is persistent and c_t is fast: z_(t+1) = F_theta(z_t,u_t). At restart times, h is preserved and c is contracted toward c_ref: c_(t+1) = (1-rho)c_ref + rho c_(t+1)^-, where rho is the retained fraction and c_(t+1)^- is the pre-restart fast state. If restarts destroy long-range correlations, the long-horizon variance changes from ballistic scaling proportional to n^2 to diffusive scaling proportional to n, with mean displacement approximately v n and variance approximately 2 D n.

## Key formulas

- $$|\psi(n)\rangle=\hat W^n|\psi(0)\rangle=\sum_{x,s}\alpha_s(x,n|0)|x,s\rangle,\qquad P(x,n)=\sum_s|\alpha_s(x,n|0)|^2.$$
- $$z_{t+1}=F_\theta(z_t,u_t),\quad z_t=(h_t,c_t),\qquad \mathcal R_\rho(h,c)=\left(h,(1-\rho)c_{\mathrm{ref}}+\rho c\right).$$
- $$\mathbb E[H_n-H_0]\simeq vn,\qquad \operatorname{Var}(H_n-H_0)\simeq 2Dn\quad\text{for }n\gg r.$$
- $$\operatorname{Var}(H_n-H_0)\propto n^2\ \text{before decorrelation},\qquad \operatorname{Var}(H_n-H_0)\propto n\ \text{after subspace restart}.$$

## Implementation notes

1. Integration point: implement this in a GRU, vanilla RNN, or diagonal state-space model for sequence forecasting or world-model rollouts. Partition the recurrent vector into h in R^d_h, which is always propagated, and c in R^d_c, which carries short-lived contextual information. Use a learned vector c_ref, or zero initially. 2. Pseudocode: compute (h,c_pre) = cell(h,c,u_t); set c = c_pre; if (t+1) modulo r equals zero, set c = (1-rho)c_ref + rho*c_pre; decode from concatenate(h,c). Sweep r in {1,2,4,8,16,32} and rho in {0.25,0.5,0.75,1}. 3. Computed versus estimated: the restart map is exact; estimate drift v and diffusion D from rollout means and variances, and estimate fast-state autocorrelation from hidden trajectories. 4. First experiment: train matched GRUs on a synthetic constant-velocity-plus-noise forecasting task and Sequential MNIST, comparing no restart, full-state reset, and subspace restart. The prediction is a crossover near n_star approximately r: for n below r, variance can be correlated or ballistic, while for n much larger than r, log variance versus log horizon approaches slope 1. Increasing r should move n_star proportionally, with measured n_star divided by r remaining within 20 percent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.