Unverified 2026

Integrable Elliptic Memory Cell

Implementation & benchmark of arXiv:2608.19393 — Outer Contact Billiards

Usefulness6/10
Difficulty4/10
Novelty4/10

Source paper: Outer Contact Billiards arXiv:2608.19393 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained recurrent transition with block-diagonal planar rotations whose angles are learned or conditioned on a slowly varying context variable. The resulting hidden-state norm and each two-dimensional block energy are exactly invariant in the ideal recurrence, preventing exploding or vanishing recurrent dynamics while retaining phase information over long horizons.

Formulas

$$h_{k+1}=\begin{pmatrix}R_2(\alpha)&0\\0&R_2(\beta)\end{pmatrix}h_k,\qquad R_2(\theta)=\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}.$$
$$\cos\alpha=\frac{1-\sigma^2}{1+\sigma^2},\quad \sin\alpha=\frac{2\sigma}{1+\sigma^2},\qquad \cos\beta=\frac{1-c^2\sigma^2}{1+c^2\sigma^2},\quad \sin\beta=\frac{-2c\sigma}{1+c^2\sigma^2}.$$
$$V(h)=h^{\mathsf T}h,\qquad V(h_{k+1})-V(h_k)=h_k^{\mathsf T}(R^{\mathsf T}R-I)h_k=0,\qquad \rho(R)=1.$$

Mathematical statement

For hidden state h=(x,y,u,v)^T in R^4, the paper's invariant-leaf map is h_{k+1}=R(alpha,beta)h_k, where R(alpha,beta)=diag(R_2(alpha),R_2(beta)) and R_2(theta)=[[cos(theta),-sin(theta)],[sin(theta),cos(theta)]]. The angles are constant on a leaf indexed by sigma^2 and satisfy cos(alpha)=(1-sigma^2)/(1+sigma^2), sin(alpha)=2sigma/(1+sigma^2), cos(beta)=(1-c^2sigma^2)/(1+c^2sigma^2), and sin(beta)=-2csigma/(1+c^2sigma^2), with c>0 and sigma=sqrt(sigma^2). Since R^T R=I, the quadratic energy V(h)=h^T h satisfies V(h_{k+1})=V(h_k), and the spectral radius is exactly one. In a neural recurrence, nonlinear input injection is permitted, but the autonomous state transition retains this orthogonal certificate.

Implementation notes

1. Integration point: use this transition as a recurrent or state-space layer before a task-specific MLP or linear readout. Partition a hidden vector of size 4m into 2m planar pairs, with one learnable angle per pair. Parameterize each angle as theta_j=pi*tanh(a_j), or generate it from a slowly varying context embedding; never form an unconstrained recurrent matrix. 2. Pseudocode: initialize h=Linear(x_0). For each input x_t, compute an input injection b_t=B x_t, set c_j=cos(theta_j) and s_j=sin(theta_j), update each pair by p'_j=c_j p_j-s_j q_j and q'_j=s_j p_j+c_j q_j, then set h_{t+1}=R h_t+b_t and y_t=C h_t. 3. Computed exactly: rotations, their orthogonality, and the predicted invariant. Estimated empirically: gradient norms, memory retention, and the influence of input injection. Monitor DeltaV_t=||h_{t+1}||^2-||h_t||^2 separately with and without b_t. 4. First cheap experiment: compare this cell with vanilla RNN and GRU at hidden width 128 on sequential MNIST and the copy-memory task at sequence lengths 1,000 and 10,000. Use identical Adam settings and input projections. The predicted signature is |DeltaV_t|/V_t below 10^-5 for the autonomous recurrence over 10,000 steps, while unconstrained RNNs show exponential growth or decay when their recurrent spectral radius differs from one. For sinusoidal input, amplitude drift should remain below 1% over 10,000 autonomous steps.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.