# Van der Pol radial-stable recurrent cell

- ID: 1107
- Canonical URL: https://synthcore.org/idea/1107/van-der-pol-radial-stable-recurrent-cell
- API JSON: https://synthcore.org/api/idea/1107.json
- API Markdown: https://synthcore.org/api/idea/1107.md
- Verification status: beats_baseline
- Source: [arXiv:2607.18337](https://arxiv.org/abs/2607.18337)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, training-dynamics
- Math tags: dynamical-systems, control-theory, linear-algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace an unconstrained linear recurrent update with a two-dimensional oscillator state per hidden feature and use amplitude-dependent damping: negative damping below a target radius and positive damping above it. The cell should preserve phase information over long sequences while preventing hidden-state explosion or collapse.

## Mathematical statement

For oscillator $k$ with state $z_k=(x_k,y_k)\in\mathbb{R}^{2}$ and radius $r_k=\|z_k\|$, use $\dot{x}_k=\omega_k y_k$ and $\dot{y}_k=-\omega_k x_k+\mu(1-r_k^2/R^2)y_k+u_k$, where $\omega_k>0$ is angular frequency, $\mu>0$ is nonlinear gain, $R>0$ is the target amplitude, and $u_k$ is an input drive. Without input, the energy $E_k=(x_k^2+y_k^2)/2$ satisfies $\dot E_k=\mu(1-r_k^2/R^2)y_k^2$. Thus the nonlinear term injects energy when $r_k<R$ and removes energy when $r_k>R$ on average over a cycle. This transfers the paper's invariant amplitude relation $(c_{1m}^{k})^2+(c_{2m}^{k})^2=r^2$ into a recurrent hidden-state constraint.

## Key formulas

- $$\dot{x}_k=\omega_k y_k,\qquad \dot{y}_k=-\omega_k x_k+\mu\left(1-\frac{x_k^2+y_k^2}{R^2}\right)y_k+u_k.$$
- $$E_k=\frac{1}{2}(x_k^2+y_k^2),\qquad \dot E_k=\mu\left(1-\frac{x_k^2+y_k^2}{R^2}\right)y_k^2+x_k u_k^{(x)}+y_k u_k^{(y)}.$$
- $$\langle\dot E_k\rangle\approx C_k\mu\left(1-\frac{\bar r_k^2}{R^2}\right),\qquad \bar r_k^2=\frac{1}{T}\int_t^{t+T}(x_k^2+y_k^2)\,dt,$$
- $$(x_{t+1},y_{t+1})=(x_t,y_t)+h\left(\omega_k y_t,\,-\omega_k x_t+\mu\left(1-\frac{x_t^2+y_t^2}{R^2}\right)y_t\right)+hB_k u_t,$$

## Implementation notes

1. Integration point: use this transition as the hidden update of an Elman-style RNN or a diagonal state-space model. Represent hidden width $d$ by $d/2$ oscillator pairs $(x_k,y_k)$, with learned input matrix $B$, positive frequencies $\omega_k$, gain $\mu$, and target radius $R$. 2. Pseudocode: compute `r2 = x*x + y*y`; compute `dx = omega*y + Bx(u)` and `dy = -omega*x + mu*(1-r2/R**2)*y + By(u)`; update `x = x + h*dx` and `y = y + h*dy`. Use RK2 if Euler discretization is unstable. 3. Computed from the paper: the amplitude-dependent gain/loss mechanism and fixed-radius interpretation. Estimated empirically: the cycle-average coefficient $C_k$, the finite-step stability region, and the effect of forcing. Log radius distributions, energy drift, gradient norms, and the recurrent Jacobian spectral radius. 4. First cheap experiment: compare this cell with a vanilla RNN, orthogonal RNN, and GRU on copy-memory, adding, and sequential-MNIST tasks at sequence lengths 100, 500, and 1000. Sweep $h\mu\in[0.01,1]$ and $R\in\{0.5,1,2\}$. The prediction is that the unforced cycle-averaged energy drift changes sign near $\bar r=R$, and hidden radii remain bounded near that value. For the undamped Euler rotation, numerical divergence should begin near $h\omega=2$; the measured boundary should agree within approximately 20%.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a readable Van der Pol oscillator cell with RK4 integration and reproducible numerical checks. The mechanism manifested: energy injection/removal changed sign exactly at r=R, observed mu-scaling matched the predicted slope (0.0768 versus 0.0768), and all tested initial radii converged to R=1.0 with approximately 6e-10 tail variation. However, the delayed-memory readout did not beat the vanilla tanh reservoir (MSE 0.4474 versus 0.3514), so there was no observed accuracy win; the idea's stated forward-Euler rotation boundary near h omega=2 is also incorrect, while the measured RK4 linear boundary was near h omega=2.8.

### Mechanism check

- Verdict: Built a readable Van der Pol oscillator cell with RK4 integration and reproducible numerical checks. The mechanism manifested: energy injection/removal changed sign exactly at r=R, observed mu-scaling matched the predicted slope (0.0768 versus 0.0768), and all tested initial radii converged to R=1.0 with approximately 6e-10 tail variation. However, the delayed-memory readout did not beat the vanilla tanh reservoir (MSE 0.4474 versus 0.3514), so there was no observed accuracy win; the idea's stated forward-Euler rotation boundary near h omega=2 is also incorrect, while the measured RK4 linear boundary was near h omega=2.8.
- Confidence: 8/10
- Limitations: Only a small fixed delayed-memory reservoir/readout comparison was tested; no trained Elman/GRU, copy-memory, adding, sequential-MNIST, FLOP-normalized study, learned frequencies, parameter sweeps over R and mu, gradient/Jacobian analysis, or GPU experiment was performed. The Euler stability claim was not confirmed because forward Euler applied to an undamped rotation is expansive for every nonzero step; the reported stability sweep concerns RK4 instead.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.02365
- Paired wins: 7/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/440/bench_report.json)
- [bench_vdp.py](https://synthcore.org/code/440/bench_vdp.py)
- [mini_compare.py](https://synthcore.org/code/440/mini_compare.py)
- [report.md](https://synthcore.org/code/440/report.md)
- [report_bench_2026-08-31T121652.md](https://synthcore.org/code/440/report_bench_2026-08-31T121652.md)
- [results.json](https://synthcore.org/code/440/results.json)
- [run_experiment.py](https://synthcore.org/code/440/run_experiment.py)
- [vdp_cell.py](https://synthcore.org/code/440/vdp_cell.py)
- [Download all files as ZIP](https://synthcore.org/download/440)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.