△ Mechanism confirmed, baseline not beaten 2026

Learned Lie-Algebra Regularizer

Implementation & benchmark of arXiv:2608.01582 — LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems

Usefulness8/10

Difficulty6/10

Novelty7/10

Dynamics Stability Generalization Sample efficiency Training dynamics Regularization Mlp World model Dynamical systems Differential geometry Linear algebra Geometry

Source paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Attach several neural vector fields to a latent representation and train them to form a closed Lie algebra rather than learning unrelated augmentation directions. The resulting generators provide data-driven continuous transformations that can be used as equivariance constraints, while bracket closure and basis-rank penalties prevent degenerate or redundant generators.

Formulas

$$[G_a,G_b](z)=J_{G_b}(z)G_a(z)-J_{G_a}(z)G_b(z)$$

$$\mathcal{L}_{\mathrm{close}}=\frac{1}{|\mathcal{B}|}\sum_{z\in\mathcal{B}}\sum_{a,b}\left\|[G_a,G_b](z)-\sum_{c=1}^{r}C_{ab}^{c}G_c(z)\right\|_2^2$$

$$\mathcal{L}_{\mathrm{Jac}}=\sum_{a,b,c}\left\|[G_a,[G_b,G_c]]+[G_b,[G_c,G_a]]+[G_c,[G_a,G_b]]\right\|_2^2$$

$$\mathcal{L}_{\mathrm{rank}}=\left\|\widehat{\Sigma}-\operatorname{diag}(\operatorname{diag}(\widehat{\Sigma}))\right\|_F^2,\quad \widehat{\Sigma}=\frac{1}{|\mathcal{B}|}\sum_{z\in\mathcal{B}}G(z)G(z)^\top,\quad G(z)=[G_1(z)\ \cdots\ G_r(z)]$$

Mathematical statement

Let z in R^d be a neural latent state and let G_a(z) in R^d, a=1,...,r, be learned vector-field generators. For vector fields, the Lie bracket is [G_a,G_b](z)=J_{G_b}(z)G_a(z)-J_{G_a}(z)G_b(z), where J_{G_a} is the Jacobian of G_a. Learn structure constants C_{ab}^c and minimize the discrepancy between each bracket and the span of the generators. Antisymmetry requires C_{ab}^c+C_{ba}^c=0, while the Jacobi residual checks the cyclic identity. A covariance-spectrum penalty prevents all generators from collapsing into the same direction.

Implementation notes

Use an encoder z=E_theta(x) and attach an MLP G_phi with r outputs, where each output G_a(z) is a d-dimensional vector. Add trainable structure constants C in R^(r x r x r). For each minibatch, evaluate all generators, compute Jacobian-vector products J_Gb G_a and J_Ga G_b with automatic differentiation, form every bracket, and regress it onto the generator span using C. Add the closure, Jacobi, antisymmetry, and rank terms to the ordinary reconstruction, prediction, or classification loss. Use exact Jacobian-vector products for small MLPs; for larger networks use a random subset of generator pairs or Hutchinson directional estimates. The Lie-bracket constraints come from the paper; the latent representation, task loss, and covariance penalty are adaptations. During the first warm-up phase, normalize each generator and stop gradients from the algebra losses into the encoder to avoid trivial zero fields. First test an autoencoder or latent predictor on trajectories from a 3D isotropic Ornstein-Uhlenbeck process, whose rotational symmetry has three nontrivial generators. Compare an unconstrained model with the regularized model and measure reconstruction, rotated-test prediction, closure residual, Jacobi residual, and singular values of the generator covariance. The prediction is that normalized closure residual falls below 10^-2 while three singular values remain nonzero; increasing closure weight past a critical value should cause a measurable rank-collapse transition.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable linear-generator MVP with exact Lie brackets, fitted structure constants, closure/Jacobi/antisymmetry checks, covariance singular values, and a regularized fitting sweep. The mechanism manifested: exact SO(3) closure/Jacobi were below 6e-33, perturbation closure MSE followed the predicted quadratic scaling with log-log slope 1.9999, and a high regularization weight produced clear rank collapse. The regularizer did not improve the already exact toy task and hurt task fit at extreme weight, so this is evidence for the mechanism rather than a general performance win.

Agent confidence: 8/10

Baseline: Task MSE 2.92e-32; closure MSE 5.72e-32; covariance singular values [0.6960, 0.6837, 0.6778].
Idea: At lambda=10: task MSE 4.19e-10; closure MSE 7.61e-16; singular values [0.6959, 0.6836, 0.6778]. At lambda=100, task MSE was 0.1914 and singular values collapsed to [0.4718, 0.0750, 0.0028]. Observed perturbation closure MSE exponent was 1.9999, matching the predicted exponent 2.

Limitations: This is a linear vector-field toy rather than the proposed encoder plus nonlinear MLP on OU trajectories. Direct vector-field Jacobi is an identity, so the diagnostic uses Jacobi residuals of fitted constant structure coefficients. No rotated-test prediction, reconstruction model, CUDA path, or larger-network automatic differentiation benchmark was tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.01582) Sep 2, 2026

No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0121
Idea mean: 0.0121
Effect (Δ): 0 (−-0.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

The custom intervention loop used CPU after the shared GPU produced a cuDNN host-allocation failure. Only the built-in dynamics track was tested, with 400 training samples, 200 test samples, 6 epochs, three learning rates, and three idea lambda settings; vision, tabular, sequence, longer training, larger generator counts, and rotated/generalization tests were not evaluated.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented the Learned Lie-Algebra Regularizer on the structurally matched dynamics track using the shared rnn_small architecture and an algebra-aware training loop with exact JVP brackets, closure, antisymmetry, Jacobi, and covariance terms. The numerical scaling check confirmed the predicted perturbation closure-MSE exponent (2.134 versus predicted 2.0), and the trained-model signature confirmed closure below 1e-2 with three nonzero covariance singular values. However, the standard task metric was exactly unchanged across all paired seeds (delta_mean=0, p=1.0), so there was no observed learning win and worked=false.

Baseline: Best lr=0.006, 6 epochs; test MSE mean 0.01214549, std 0.00545378; per-seed [0.02137684, 0.01083950, 0.01003816, 0.01721210, 0.00647046, 0.01678092, 0.01028555, 0.00416037]. Baseline sweep means were 0.15673569 at lr=0.001, 0.02257082 at lr=0.003, and 0.01214549 at lr=0.006.
Idea: Best lambda=0.0003 and rank weight=0.001 at lr=0.006, 6 epochs; test MSE mean 0.01214549, std 0.00545378, with identical per-seed results to baseline. Seed-0 signature: closure 3.415e-05, Jacobi residual 1.541e-08, covariance singular values [0.0075458, 0.0065326, 0.0058982].
Paired comparison: delta_mean=0.0, p_value=1.0, idea_wins=0/8, verdict=no measurable effect

Benchmark result