Unverified 2026

Third-Order Nilpotent Memory Cell

Implementation & benchmark of arXiv:2607.13286 — Non-vanishing of multiple correlation sequences

Usefulness6/10

Difficulty7/10

Novelty8/10

Memory Stability Accuracy Memory Rnn Ssm World model Training dynamics Dynamical systems Geometry Harmonic analysis Linear algebra

Source paper: Non-vanishing of multiple correlation sequences arXiv:2607.13286 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace or augment an RNN or state-space model hidden state with coordinates on a bounded 3-step nilpotent group. The first layer stores ordinary features, the second layer stores pairwise commutator memory, and the third layer stores nested commutators that can preserve three-time dependencies invisible to first- and second-order summaries. Layered reduction keeps the state bounded while retaining the algebraic interaction structure.

Formulas

$$\operatorname{BCH}(a,b)=a+b+\frac12[a,b]+\frac1{12}[a,[a,b]]-\frac1{12}[b,[a,b]],\qquad [\mathfrak g,[\mathfrak g,[\mathfrak g,\mathfrak g]]]=0.$$

$$z_{t+1}=\operatorname{red}\!\left(\operatorname{BCH}(z_t,\,W x_t+b)\right),\qquad z_t=(z_t^{(1)},z_t^{(2)},z_t^{(3)}),$$

$$C(n)=\frac1T\sum_{t=1}^{T-2n}f_0(z_t)f_1(z_{t+n})f_2(z_{t+2n}),\qquad C_{\perp}(n)=\frac1T\sum_t\prod_{j=0}^{2}\left(f_j(z_{t+jn})-\mathbb E[f_j\mid z^{(1)},z^{(2)}]\right).$$

$$\widetilde{\operatorname{red}}(s)=\tau^{-1}\!\left(\operatorname{red}(\tau(s))\right)\in[0,1)^d.$$

Mathematical statement

Let $\mathfrak g=V_1\oplus V_2\oplus V_3$ be a step-3 nilpotent Lie algebra, meaning $[V_i,V_j]\subseteq V_{i+j}$ and $V_k=0$ for $k>3$. For a state $z_t\in\mathfrak g$ and input $u_t\in V_1$, use the truncated Baker–Campbell–Hausdorff product $z_{t+1}=\operatorname{red}(\operatorname{BCH}(z_t,u_t))$, where $\operatorname{BCH}(a,b)=a+b+\frac12[a,b]+\frac1{12}[a,[a,b]]-\frac1{12}[b,[a,b]]$ because all commutators of depth four vanish. The state is reduced into a fundamental domain $[0,1)^d$ by multiplying by a lattice element, as in the paper's generalized-polynomial reduction map. The key structural signature is that a triple correlation $C(n)=\mathbb E[f_0(z_t)f_1(z_{t+n})f_2(z_{t+2n})]$ can remain nonzero even when each $f_i$ has zero conditional expectation with respect to the projection onto $V_1\oplus V_2$.

Implementation notes

1. Integration point: implement this as a recurrent cell for sequence prediction, either replacing a vanilla tanh RNN or concatenating its nilpotent state with a conventional hidden state. Use a small algebra, for example four first-layer generators, six learned second-layer commutator coordinates, and one or several third-layer coordinates; a smaller free step-3 algebra is sufficient for an MVP. Represent the Lie bracket by fixed sparse tensors $B_{ab}^c$, with nonzero entries only from layer 1 to layers 2 and 3. 2. Pseudocode: initialize $z=0$; for each input $x_t$, set $u=W x_t+b$ in layer 1; compute $q=z+u+0.5\,\operatorname{bracket}(z,u)+(1/12)\,\operatorname{bracket}(z,\operatorname{bracket}(z,u))-(1/12)\,\operatorname{bracket}(u,\operatorname{bracket}(z,u))$; apply layered reduction by subtracting integer lattice coordinates from layer 1, then recomputing the induced layer-2 and layer-3 coordinate corrections using the same bracket tensors; output $y_t=R[z^{(1)},z^{(2)},z^{(3)}]$. Train with the task loss plus a small optional penalty on bracket-tensor norm. 3. Computed from the paper: the step-3 BCH truncation, layer ordering, lattice reduction, and the prediction that third-layer observables are not determined by the 2-step projection. Estimated empirically: bracket tensors, readout, conditional projections, and correlation amplitudes. 4. First cheap experiment: train the cell and a matched tanh-RNN and a 2-step BCH cell on synthetic sequences generated by a fixed step-3 nilpotent process, with targets depending on a third-layer coordinate after 100–1000 steps. Measure test error and $C(n)$ for readouts residualized against $(z^{(1)},z^{(2)})$. The quantitative prediction is that the 3-step cell maintains a statistically significant nonzero $C_{\perp}(n)$ over the process's correlation horizon, while the 2-step and tanh baselines' residual triple correlation decays toward zero. Also sweep the input scale $\sigma$: because the state is reduced to a compact domain, hidden-state magnitude should remain bounded independently of sequence length, whereas an unreduced BCH cell should show growth or numerical instability.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.