△ Mechanism confirmed, baseline not beaten 2026

Lie-Group Lyapunov Recall Dynamics

Implementation & benchmark of arXiv:2608.08226 — High-Capacity Generalized Hopfield Networks

Usefulness7/10

Difficulty6/10

Novelty7/10

Dynamics Stability Memory Accuracy Rnn Optimizer Training dynamics Memory Dynamical systems Control theory Geometry Linear algebra

Source paper: High-Capacity Generalized Hopfield Networks arXiv:2608.08226 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use dissipative dynamics directly on the SU(d) manifold instead of unconstrained Euclidean recurrent updates. A Riemannian gradient or damped Landau-Lifshitz-Gilbert-like flow preserves the unitary constraint and supplies an explicit Lyapunov certificate: the associative-memory energy should decrease monotonically until the state reaches a recalled attractor.

Formulas

H_i = sum_j J_ij(U_j), skew_0(A) = 0.5(A - A dagger) - Tr(A - A dagger) I/(2d)

dU_i/dt = -eta grad_i E = U_i Omega_i, Omega_i in su(d)

dE/dt = -eta sum_i ||grad_i E||_F^2 <= 0

U_i(k+1) = U_i(k) exp(-eta Omega_i(k))

Mathematical statement

For U_i in SU(d), define the local field H_i = sum_j J_ij(U_j). The tangent-space projection uses skew_0(A) = 0.5(A - A dagger) - Tr(A - A dagger) I/(2d), which is traceless and skew-Hermitian. Let Omega_i be the projected tangent force derived from the negative energy gradient. The continuous update dU_i/dt = U_i Omega_i remains on SU(d), because U_i dagger dU_i/dt belongs to the Lie algebra su(d). For gradient descent with learning rate eta, the energy obeys dE/dt = -eta sum_i ||grad_i E||_F^2, which is nonpositive. A discrete exponential-map update preserves the group constraint much better than an unconstrained Euler step.

Implementation notes

Integration point: use this as the recurrent inference solver for the SU(d) memory layer, or as a constrained optimizer for a trainable associative-memory energy. Do not update real and imaginary matrix entries independently; update each state with a matrix exponential so unitarity is preserved. Pseudocode: compute H_i = N^(-1) sum over j and mu of M_i^mu (M_j^mu) dagger U_j; calculate the Euclidean energy gradient; multiply by U_i dagger; project the result with skew_0; set U_i to U_i exp(-eta Omega_i); then evaluate energy, overlap, and the constraint error ||U_i dagger U_i - I||_F. If energy increases, halve eta and retry. The Lie-group flow and Lyapunov identity come from the paper; estimate the local gradient Lipschitz constant L from successive gradients and use eta below 2/L as a conservative discrete-time ceiling. First cheap experiment: run N = 128, d = 3, and P = 0.5N on random memories and corrupted RGB-image patches. Compare exponential-map, naive matrix Euler, and Euclidean projected-gradient updates. The predicted signature is monotonic energy decay and machine-precision unitarity for the exponential-map method; fixed-step instability should emerge near eta = 2/L, while backtracking should remove that instability and converge to a high-overlap attractor.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built an SU(3) associative-memory solver using projected su(d) forces and exponential-map updates, with Euler and QR-projected baselines. The three Stage-1 mechanism predictions were confirmed: the Lyapunov finite-difference ratio approached 1.00000002, coupling scaling followed the predicted c^4 law within 7e-8 relative error, and exponential-map unitarity stayed at 1.7e-15–2.9e-15 while Euler error grew to 1.50. The proposed local 2/L discrete ceiling was not quantitatively reliable: monotonicity failed around 0.35*(2/L), but backtracking restored monotonic descent.

Agent confidence: 9/10

Baseline: Naive tangent Euler, eta=0.35 for 30 steps: max ||U†U-I||_F=1.50; final energy=-123.22, but states leave SU(3). QR-projected Euler: constraint error=2.22e-15, final energy=-92.3645, overlap=0.4883.
Idea: Exponential-map SU(3), eta=0.35 for 30 steps: max unitarity error=2.93e-15, determinant error=1.56e-15, final energy=-92.3698, overlap=0.4889. Lyapunov derivative=-24.745543 versus prediction -24.745543, ratio 1.00000002; backtracking was monotone with energy drop 63.5653.

Limitations: This is a small N=32,d=3,P=16 synthetic random-memory test, not the requested N=128 RGB-patch experiment. No learned memory, CUDA benchmark, multi-seed statistics, or rigorous global Lipschitz estimate was tested; the local 2/L estimate did not predict the observed transition.

How to run: python3 stage1_verify.py && /home/maxwelhelp/main/bin/python3 lie_group_recall.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.08226) Sep 2, 2026

Ahead of the baseline (−8.9%) but not statistically significant (p=0.6103, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.001
Idea mean: 0.0009
Effect (Δ): -0.0001 (−8.9%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.6103 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±35.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 4 configs

Limitations:

Only the built-in dynamics task, rnn_small model, 400 training samples, 200 test samples, 15 epochs, and 8 seeds were tested. The implementation uses real orthogonal gate-wise retraction as a practical analogue of SU(d); complex SU(d) recurrent states and larger models were not evaluated.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

The Lie-group-inspired recurrent update was evaluated on the registered dynamics track with the shared rnn_small architecture, tuned baseline, shared hyperparameter grid, and 8 paired seeds. The idea reduced mean MSE from 0.00099409 to 0.00090552, but the paired permutation test gave p=0.61035, so this is not a significant win. The trained-model mechanism signature was confirmed: maximum gate orthogonality error was 9.24e-06, below the 1e-05 target.

Baseline: Best Adam baseline lr=0.006, weight_decay=0.0; mean test MSE=0.0009940863528754562, std=0.0002634151189555855.
Comparison: delta_mean=-8.856166823534295e-05, idea_wins=3/8, permutation p_value=0.61035, verdict=no significant win.
Idea: Best Lie-group-retracted recurrent training lr=0.006, weight_decay=0.0; mean test MSE=0.0009055246846401133, std=0.0003078280604952229.
Mechanism signature: predicted_max_constraint_error_below=1e-05; observed_max_gate_orthogonality_error=9.236763617082033e-06; confirmed=true.

Benchmark result