# BT-Critical Long-Memory Initialization

- ID: 2554
- Canonical URL: https://synthcore.org/idea/2554/bt-critical-long-memory-initialization
- API JSON: https://synthcore.org/api/idea/2554.json
- API Markdown: https://synthcore.org/api/idea/2554.md
- Verification status: unverified
- Source: [arXiv:2608.19018](https://arxiv.org/abs/2608.19018)
- Category: architecture
- Solves: accuracy, stability
- ML areas: rnn, ssm, initialization, scheduler
- Math tags: dynamical-systems, bifurcations, spectral-theory, linear-algebra, control-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 8/10

## Idea description

Construct a recurrent or state-space layer whose equilibrium Jacobian is placed near a nondegenerate Bogdanov–Takens point, then use a small unfolding parameter to move between damped, oscillatory, and slowly relaxing regimes. Unlike eigenvalue-only initialization near one, this controls both the double-zero center structure and the quadratic nonlinear coefficients that determine the local phase portrait.

## Mathematical statement

For a parameterized neural vector field $\dot z=f_{\theta,\mu}(z)$, choose an equilibrium $z_*$ and require $J=D_zf_{\theta,\mu_0}(z_*)$ to have a nilpotent two-dimensional center block and an invertible transverse block. Define $\delta=e_n(J)/e_{n-2}(J)$ and $\tau=(e_{n-1}(J)e_{n-2}(J)-e_n(J)e_{n-3}(J))/e_{n-2}(J)^2$. The BT conditions are $\delta=0$ and $\tau=0$, with $e_{n-2}\neq0$. If $q_0\in\ker J$, the quadratic normal-form coefficients are $a=-\frac12D_{q_0}\delta$ and $b=D_{q_0}\tau$. Nondegeneracy requires $ab\neq0$. The initialization should therefore minimize $\delta^2+\tau^2$ while maintaining a safe transverse denominator and nonzero bounded coefficients, with $\mu$ retained as an explicit unfolding coordinate.

## Key formulas

- $$\delta=\frac{e_n(J)}{e_{n-2}(J)},\qquad \tau=\frac{e_{n-1}(J)e_{n-2}(J)-e_n(J)e_{n-3}(J)}{e_{n-2}(J)^2},\qquad e_{n-2}(J)\neq0.$$
- $$a=-\frac12D_{q_0}\delta,\qquad b=D_{q_0}\tau,\qquad ab\neq0.$$
- $$\dot u_1=u_2+\mathcal{O}(\|u\|^3),\qquad \dot u_2=au_1^2+bu_1u_2+\mathcal{O}(\|u\|^3).$$
- $$L_{\mathrm{critical}}=\lambda_\delta\delta^2+\lambda_\tau\tau^2+\lambda_e\,[\max(0,\epsilon-|e_{n-2}|)]^2+\lambda_a[\max(0,a_{\min}^2-a^2)]^2.$$

## Implementation notes

Integrate this at initialization of a small continuous-time RNN or structured state-space layer. Use a vector field such as $f(z)=Az+W_2\tanh(W_1z+c)$ and expose a scalar gain $\mu$ multiplying the designated two-dimensional center block. First choose parameters, solve for a reference equilibrium $z_*$, compute $J$, and calculate its characteristic invariants. Estimate a normalized null vector $q_0$ using the smallest right singular vector. Use JVPs through the Jacobian and invariant calculation to obtain $D_{q_0}\delta$ and $D_{q_0}\tau$. Optimize the initialization-only objective $L_{critical}$ for 100--500 steps while freezing task weights; reject updates for which $|e_{n-2}|<\epsilon$, then train normally while retaining $\mu$ as a learned or scheduled unfolding parameter. Compare vanilla initialization, eigenvalue-only near-critical initialization, and this BT initialization on a delayed-copy and damped-sinusoid task. Sweep $\mu$ and measure memory time, oscillation frequency, and rollout stability. The falsifiable prediction is that relaxation time increases as $|\delta|+|\tau|$ decreases, while the transition from monotone decay to oscillatory or growing trajectories occurs near the simultaneous invariant zero, not merely when the largest eigenvalue approaches zero. Fit local trajectories to the quadratic normal form and require recovered coefficients to agree with the invariant estimates within 20 percent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.