# Ghost-State Adaptive Recurrent Cell

- ID: 2336
- Canonical URL: https://synthcore.org/idea/2336/ghost-state-adaptive-recurrent-cell
- API JSON: https://synthcore.org/api/idea/2336.json
- API Markdown: https://synthcore.org/api/idea/2336.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.15300](https://arxiv.org/abs/2608.15300)
- Category: architecture
- Solves: stability, accuracy, memory
- ML areas: rnn, ssm, mlp
- Math tags: dynamical-systems, bifurcations, singular-perturbation, control-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a single recurrent state update with fast feature relaxation, activity evolution, and a slow adaptive state that modulates the activity vector field. Tune the activity subsystem near a controllable saddle-node so that it retains a useful transient regime for a predictable number of steps, enabling delayed switching and long-horizon memory without requiring a large hidden state.

## Mathematical statement

Use a fast–slow recurrent system with input \(u_k\), fast state \(X_k\), activity state \(A_k\), and adaptive state \(B_k\). Let \(\tau_X\ll\tau_A\ll\tau_B\), and update \(X\) toward an encoder equilibrium, while \(A\) follows a parameterized vector field \(f(A;B,u)\) and \(B\) adapts slowly. For a scalar activity coordinate \(z\) near a saddle-node, the normal form is \(\dot z=\mu+z^2\), where \(\mu\) is the signed distance from the fold. For \(\mu>0\), passage through the bottleneck takes \(T_{\mathrm{ghost}}\approx\pi/\sqrt{\mu}\). If the adaptive state causes a transverse sweep \(\mu(t)=\varepsilon t\), with slow-passage rate \(\varepsilon>0\), the delayed transition occurs on the scale \(T_{\mathrm{delay}}=O(\varepsilon^{-1/3})\). The architecture should therefore expose \(\mu\) and \(\varepsilon\) as measurable control quantities rather than hiding them inside arbitrary recurrent weights.

## Key formulas

- $$\tau_X\dot X=E(u)-X,\qquad \tau_A\dot A=f_\theta(A;B,u),\qquad \tau_B\dot B=\varepsilon g_\theta(B,A,u),\qquad 0<\tau_X\ll\tau_A\ll\tau_B.$$
- $$\dot z=\mu+z^2,\qquad T_{\mathrm{ghost}}=\int_{-\infty}^{\infty}\frac{dz}{\mu+z^2}=\frac{\pi}{\sqrt{\mu}},\quad \mu>0.$$
- $$\mu(t)=\varepsilon t\quad\Longrightarrow\quad T_{\mathrm{delay}}=O(\varepsilon^{-1/3}).$$
- $$X_{k+1}=(1-\alpha_X)X_k+\alpha_XE_\theta(u_k),\quad A_{k+1}=A_k+\alpha_A f_\theta(A_k;B_k,X_{k+1},u_k),\quad B_{k+1}=B_k+\alpha_B g_\theta(B_k,A_k,u_k).$$

## Implementation notes

Integrate this as a replacement for the hidden-state update in a GRU-like RNN or diagonal state-space model. Use three vectors: a fast state X for current input encoding, an activity state A for the recurrent representation, and a slow state B for context or adaptation. Set alpha_X near 1, alpha_A substantially smaller, and alpha_B at least 5–20 times smaller than alpha_A. Implement f with a residual MLP, but constrain one scalar channel z of A to have an explicit near-fold form f_z=mu(B,u)+z^2-cz, where c can be fixed positive or learned with softplus; the remaining channels use a standard bounded residual update. Define mu(B,u)=w_mu^T[B,u]+b_mu and estimate the local fold by solving f_z=0 and partial_z f_z=0, or use the analytically known fold of the normal form. Pseudocode is: compute X=(1-alpha_X)X+alpha_X E(u); compute mu; update z and the remaining A channels; update B with the slow rule; predict from [X,A,B]. The paper-derived quantities are the time-scale ordering, inverse-square-root ghost law, and epsilon-to-the-minus-one-third slow-passage law. Empirically estimate mu and epsilon from trajectories by finite differences and regress dwell time against them. First experiment: train this cell and a matched GRU on synthetic sequences whose regime switches after a delayed cue, then on sequential MNIST or permuted sequential MNIST. Sweep fixed positive mu values and measure time spent in a small interval around z=0; the predicted signature is T proportional to mu^(-1/2), with a log-log slope near -0.5. Then sweep alpha_B and measure transition delay while B drives mu through zero; the predicted slope is approximately -1/3 against epsilon=alpha_B, within broad finite-size deviations. Compare long-horizon error and hidden-state norm against the GRU.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and numerically verified the Ghost-State mechanism. The fixed-mu ghost passage produced a log-log slope of -0.504 versus the predicted -0.5, with maximum numerical-vs-analytic relative error 7.9e-8; slow passage produced -0.352 versus the predicted -1/3. The mechanism manifested clearly, but the small fixed-dynamics GhostCell underperformed the matched GRU on delayed-cue accuracy (0.656 vs 0.850), so no task-level win was observed.

### Mechanism check

- Verdict: Built and numerically verified the Ghost-State mechanism. The fixed-mu ghost passage produced a log-log slope of -0.504 versus the predicted -0.5, with maximum numerical-vs-analytic relative error 7.9e-8; slow passage produced -0.352 versus the predicted -1/3. The mechanism manifested clearly, but the small fixed-dynamics GhostCell underperformed the matched GRU on delayed-cue accuracy (0.656 vs 0.850), so no task-level win was observed.
- Confidence: 9/10
- Limitations: The recurrent comparison uses a fixed, minimally trained GhostCell readout rather than end-to-end optimization of all cell parameters, and only one synthetic delayed-cue task was tested. No sequential MNIST, hyperparameter sweep, FLOP/latency benchmark, or long-horizon robustness evaluation was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [add_gru.py](https://synthcore.org/code/806/add_gru.py)
- [bench_report.json](https://synthcore.org/code/806/bench_report.json)
- [ghost_bench.py](https://synthcore.org/code/806/ghost_bench.py)
- [ghost_experiment.py](https://synthcore.org/code/806/ghost_experiment.py)
- [report.md](https://synthcore.org/code/806/report.md)
- [report_bench_2026-09-01T110141.md](https://synthcore.org/code/806/report_bench_2026-09-01T110141.md)
- [results.json](https://synthcore.org/code/806/results.json)
- [Download all files as ZIP](https://synthcore.org/download/806)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.