# Invariant-Sphere Recurrent State

- ID: 2936
- Canonical URL: https://synthcore.org/idea/2936/invariant-sphere-recurrent-state
- API JSON: https://synthcore.org/api/idea/2936.json
- API Markdown: https://synthcore.org/api/idea/2936.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.28223](https://arxiv.org/abs/2608.28223)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, bifurcations, control-theory, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace an unconstrained recurrent transition by a ring-coupled cubic vector field whose radial component drives hidden states toward a prescribed sphere. The angular component remains trainable and can encode information, while the radial Lyapunov dynamics suppress exploding and vanishing state norms during long rollouts.

## Mathematical statement

Let $h\in\mathbb{R}^{n}$ be the recurrent state, $r=\lVert h\rVert$, and define the ring-coupled homogeneous cubic field $Q(h)$ by $Q_i(h)=q(h_i,h_{i+1})$, with cyclic indexing $h_{n+1}=h_1$ and $q(y,z)=ay^3+by^2z+cyz^2+dz^3$. The continuous recurrent dynamics are $\dot h=\lambda h+Q(h)$, where $\lambda>0$. If the cubic radial form satisfies $\langle u,Q(u)\rangle=-\kappa$ for every unit vector $u$ or approximately satisfies this with bounded variation, then $V(h)=\frac12\lVert h\rVert^2$ obeys $\dot V=\lambda r^2-\kappa r^4$, giving the attracting radius $r_*=\sqrt{\lambda/\kappa}$. More generally, if $\langle u,Q(u)\rangle\leq-\kappa_{\min}<0$ on the unit sphere, then sufficiently large states contract radially. At the pole $u=e_1$, the paper identifies a tangential eigenvalue proportional to $c+1$, so $c=-1$ is a local stability boundary that can be monitored as a trainable bifurcation parameter.

## Key formulas

- $$Q_i(h)=q(h_i,h_{i+1}),\qquad q(y,z)=ay^3+by^2z+cyz^2+dz^3,\qquad \dot h=\lambda h+Q(h).$$
- $$\dot V(h)=\left\langle h,\lambda h+Q(h)\right\rangle=\lambda r^2+r^4\left\langle u,Q(u)\right\rangle,\qquad h=ru,\quad \lVert u\rVert=1.$$
- $$\left\langle u,Q(u)\right\rangle=-\kappa\ \Longrightarrow\ \dot r=\lambda r-\kappa r^3,\qquad r_*=\sqrt{\lambda/\kappa},\qquad \left.\frac{d\dot r}{dr}\right|_{r_*}=-2\lambda.$$
- $$P(u)=Q(u)-\langle Q(u),u\rangle u,\qquad \mu_{\mathrm{pole}}=c+1;\quad c=-1\ \text{is the predicted local tangential stability transition at }u=\pm e_1.$$

## Implementation notes

1. Integration point: use this as the hidden transition of an RNN or state-space layer. Partition a hidden vector into a cyclic ring, compute $Q_i=ah_i^3+bh_i^2h_{i+1}+ch_i h_{i+1}^2+dh_{i+1}^3$, and add input injection separately. Set $\lambda>0$ and initialize coefficients so the sampled radial form is negative. Use a sufficiently small solver step, or compare against an explicit projection to radius $r_*=\sqrt{\lambda/\kappa}$. 2. Pseudocode: `u=h/(norm(h)+eps); Q=ring_cubic(h); hdot=lambda*h+Q+W_in*x_t; h=h+dt*hdot; loss=task_loss`. Add `radial_penalty=(mean(inner)+kappa)^2+variance(inner)`, where `inner=dot(u,Q(u))` over random unit-vector probes. Optionally parameterize $c$ around the predicted boundary $c=-1$. 3. Compute directly from the mechanism the cubic field, Lyapunov derivative, predicted radius, and pole eigenvalue. Estimate empirically the radial coefficient $\kappa=-\operatorname{mean}_{u\sim S^{n-1}}\langle u,Q(u)\rangle$, its variance, the recurrent Jacobian spectral radius, and norm relaxation rate. 4. First experiment: compare equal-width vanilla tanh RNN, orthogonal RNN, and this invariant-sphere RNN on copy-memory and adding tasks at sequence lengths 100, 500, and 1000, using equal parameter counts and optimizer budgets. Include ablations removing the radial penalty and replacing cubic ring coupling by a dense transition. 5. Pre-register the signature: with zero input, the hidden norm should converge to $r_*$ and satisfy $|r_t-r_*|\approx C\exp(-2\lambda t)$ in continuous-time units. Sweeping $c$ should change the pole-direction multiplier at $c=-1$. For Euler integration, the radial linearization predicts a step-size boundary near $dt_c\approx 1/\lambda$. Reject the transfer if the radial form is positive on more than 1 percent of sampled sphere directions, if the measured relaxation rate differs from $2\lambda$ by more than 30 percent, or if norm variance is not reduced by at least 50 percent without improving long-horizon accuracy.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a ring-coupled cubic recurrent state and verified its stability numerically. The field had zero positive radial directions across 20,000 probes, drove initial norms spanning 0.66–17.92 to approximately 2.80–2.83, and the exact scalar case converged to radius 1 with measured relaxation slope -2.31 versus the predicted -2.0. In a small delayed-bit task, the sphere RNN achieved test MSE 0.00013 versus 0.152 for the tanh baseline, a promising but preliminary signal rather than a definitive general result.

### Mechanism check

- Verdict: Built a ring-coupled cubic recurrent state and verified its stability numerically. The field had zero positive radial directions across 20,000 probes, drove initial norms spanning 0.66–17.92 to approximately 2.80–2.83, and the exact scalar case converged to radius 1 with measured relaxation slope -2.31 versus the predicted -2.0. In a small delayed-bit task, the sphere RNN achieved test MSE 0.00013 versus 0.152 for the tanh baseline, a promising but preliminary signal rather than a definitive general result.
- Confidence: 7/10
- Limitations: Only one seed and one short delayed-bit task were tested; no copy/addition benchmark at lengths 500–1000, equal-FLOP or equal-optimization studies, orthogonal-RNN baseline, radial-penalty ablation, dense-coupling ablation, Jacobian spectral measurements, or systematic c and step-size sweeps were performed. The multidimensional field has a direction-dependent radial coefficient rather than the ideal constant-kappa form.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/1138/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/1138/bench_stage2.py)
- [report.md](https://synthcore.org/code/1138/report.md)
- [report_bench_2026-09-02T122613.md](https://synthcore.org/code/1138/report_bench_2026-09-02T122613.md)
- [results.json](https://synthcore.org/code/1138/results.json)
- [run_experiment.py](https://synthcore.org/code/1138/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/1138)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.