△ Mechanism confirmed, baseline not beaten 2026

Graded Levy-area recurrent state

Implementation & benchmark of arXiv:2608.08756 — Infinite-Dimensional Levy Area: Probability-Selected Critical Geometry and Sharp Spectral Selection

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Accuracy Scalability Memory Rnn Ssm Embedding Algebra Geometry Linear algebra

Source paper: Infinite-Dimensional Levy Area: Probability-Selected Critical Geometry and Sharp Spectral Selection arXiv:2608.08756 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace additive recurrent pooling with a graded state containing the current feature increment, an antisymmetric order-sensitive area matrix, and an optional symmetric quadratic-variation accumulator. Compose chunks using the paper's exact group law, allowing a sequence model to retain compressed pairwise ordering information without explicitly forming all token pairs.

Formulas

$$X=\frac{1}{2}x^{\otimes 2}+A-\frac{1}{2}Q,\qquad A^{*}=-A,\qquad Q^{*}=Q,$$

$$x\wedge_{\mathrm{op}}y:=x\otimes y-y\otimes x,$$

$$(x,A,Q)\star(y,B,R)=\left(x+y,\,A+B+\frac{1}{2}x\wedge_{\mathrm{op}}y,\,Q+R\right),$$

$$\delta_{\lambda}(x,A,Q)=(\lambda x,\lambda^{2}A,\lambda^{2}Q).$$

Mathematical statement

The paper represents a finite second-order state as X=(x,A,Q), where x is a first-order vector, A is an antisymmetric operator or matrix satisfying A^*=-A, and Q is a symmetric operator satisfying Q^*=Q. Its sequential composition is (x,A,Q) star (y,B,R)=(x+y, A+B+1/2(x wedge_op y), Q+R), with x wedge_op y := x tensor y - y tensor x. Thus the area update adds the cross-interaction between two consecutive chunks, while Q is additive. The dilation delta_lambda(x,A,Q)=(lambda x,lambda^2 A,lambda^2 Q) states that first-order and second-order channels should use different normalization scales. The representation is useful because A changes sign when the order of x and y is reversed, unlike a symmetric covariance or sum.

Implementation notes

Integrate this at the recurrent or state-space memory update, before the usual gated transition or output projection. Let each token embedding h_t in R^d be converted into a bounded increment x_t=P tanh(W h_t), where P projects to a small state dimension k, typically 16-64. Maintain s_t=(u_t,A_t,Q_t), with u_t in R^k, A_t in R^{k x k} constrained to be skew-symmetric, and Q_t either diagonal or low-rank symmetric. For a chunk increment (v,B,R), use the exact update u_new=u+v, A_new=A+B+0.5*(u v^T-v u^T), and Q_new=Q+R. In the simplest token-level version set v=x_t, B=0, and R=diag(x_t squared); in a chunked version compute B and R recursively inside each chunk and merge chunks with the same formula. Feed [u_new, vec_skew(A_new), diag(Q_new)] through LayerNorm and a learned linear projection to produce the recurrent output. Normalize u by sqrt(t), A by t, and Q by t, matching the dilation law that second-order coordinates scale quadratically. Use a rank-r factorization A=UV^T-VU^T when k is larger, or maintain only a random/sketched bilinear map to avoid O(k^2) memory. Pseudocode: initialize u,A,Q=0; for each x, compute v=encoder(x), update A=A+0.5*(outer(u,v)-outer(v,u)), update u=u+v, update Q=Q+diag(v*v), and read out concat(u/sqrt(t), A/t, Q/t). The mathematical update is exact; the empirical choices are projection dimension, rank, gating, and normalization. First test on permuted-MNIST, ListOps, and a small character-level language-model benchmark using a GRU, diagonal SSM, and additive pooling as baselines. Compare equal hidden-state bytes and equal FLOPs. Success is higher accuracy or perplexity at the same memory, particularly on tasks where AB and BA must produce different outputs; also verify that chunked inference matches token-by-token inference up to floating-point error.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built the graded Levy-area state with exact skew-symmetric area, diagonal quadratic variation, dilation-compatible normalization, and chunk composition. All quantitative mechanism checks matched prediction exactly: reversal area scaled as |a|/2 and canceled under reversal, repeated-pair area accumulated as n/2, dilation ratios were 1.0 for u, A, and Q, and chunking error was 2.7e-15. The small benchmark did not show an accuracy win: Levy and additive pooling both reached 0.512, while Levy was slower and used more parameters.

Agent confidence: 9/10

Baseline: Additive pooling: test accuracy 0.5117, 82 parameters, 0.268 s; GRU: 0.4844 accuracy, 514 parameters, 1.456 s.
Idea: Graded Levy state: test accuracy 0.5117, 154 parameters, 2.722 s; reversal sweep exact, linear accumulation exact, dilation ratios 1.0, chunk error 2.7e-15.

Limitations: Only a tiny synthetic order-sensitive classification task was tested, not permuted-MNIST, ListOps, or language modeling. The benchmark used full-batch training and did not equalize hidden-state bytes or FLOPs; CUDA cuDNN failed on the shared device, so the reported benchmark used CPU fallback.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.08756) Sep 2, 2026

Ahead of the baseline (−2.6%) but not statistically significant (p=0.22, wins 7 / 8 paired seeds).

Benchmark: Sequence modeling
Model: custom_recurrent_state
Paired seeds: 8
Baseline mean: 1.0421
Idea mean: 0
Effect (Δ): -0.027 (−2.6%; negative = idea better)
Wins: 7 / 8 paired seeds
p-value: 0.22 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in sequence track was tested, using 15 epochs and 400 training/400 test examples. Other tracks were not tested. The idea model had 185 parameters versus 65 for baseline; equal-parameter and equal-FLOP comparisons were not performed.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

The graded Levy-area recurrent state was implemented and evaluated on the structurally matched sequence forecasting track with canonical training, matched learning-rate sweeps, and 8 paired seeds. Mean test MSE improved from 1.04215 to 1.01520 with 7/8 wins, but the paired permutation test was non-significant (p=0.21995), so no significant win was established. The trained-model reversal signature confirmed the predicted area contribution quantitatively.

Baseline: best lr=0.01; mean test MSE 1.0421477407; std 0.1010176444
Idea: best lr=0.01; mean test MSE 1.0151974857; std 0.0879008312; paired delta -0.0269502550; 7/8 wins; p=0.21995; mechanism confirmed

Benchmark result