# Learned Lie-Algebra Regularizer - ID: 1676 - Canonical URL: https://synthcore.org/idea/1676/learned-lie-algebra-regularizer - API JSON: https://synthcore.org/api/idea/1676.json - API Markdown: https://synthcore.org/api/idea/1676.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2608.01582](https://arxiv.org/abs/2608.01582) - Category: dynamics - Solves: stability, generalization, sample-efficiency - ML areas: training-dynamics, regularization, mlp, world-model - Math tags: dynamical-systems, differential-geometry, linear-algebra, geometry - Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10 ## Idea description Attach several neural vector fields to a latent representation and train them to form a closed Lie algebra rather than learning unrelated augmentation directions. The resulting generators provide data-driven continuous transformations that can be used as equivariance constraints, while bracket closure and basis-rank penalties prevent degenerate or redundant generators. ## Mathematical statement Let z in R^d be a neural latent state and let G_a(z) in R^d, a=1,...,r, be learned vector-field generators. For vector fields, the Lie bracket is [G_a,G_b](z)=J_{G_b}(z)G_a(z)-J_{G_a}(z)G_b(z), where J_{G_a} is the Jacobian of G_a. Learn structure constants C_{ab}^c and minimize the discrepancy between each bracket and the span of the generators. Antisymmetry requires C_{ab}^c+C_{ba}^c=0, while the Jacobi residual checks the cyclic identity. A covariance-spectrum penalty prevents all generators from collapsing into the same direction. ## Key formulas - $$[G_a,G_b](z)=J_{G_b}(z)G_a(z)-J_{G_a}(z)G_b(z)$$ - $$\mathcal{L}_{\mathrm{close}}=\frac{1}{|\mathcal{B}|}\sum_{z\in\mathcal{B}}\sum_{a,b}\left\|[G_a,G_b](z)-\sum_{c=1}^{r}C_{ab}^{c}G_c(z)\right\|_2^2$$ - $$\mathcal{L}_{\mathrm{Jac}}=\sum_{a,b,c}\left\|[G_a,[G_b,G_c]]+[G_b,[G_c,G_a]]+[G_c,[G_a,G_b]]\right\|_2^2$$ - $$\mathcal{L}_{\mathrm{rank}}=\left\|\widehat{\Sigma}-\operatorname{diag}(\operatorname{diag}(\widehat{\Sigma}))\right\|_F^2,\quad \widehat{\Sigma}=\frac{1}{|\mathcal{B}|}\sum_{z\in\mathcal{B}}G(z)G(z)^\top,\quad G(z)=[G_1(z)\ \cdots\ G_r(z)]$$ ## Implementation notes Use an encoder z=E_theta(x) and attach an MLP G_phi with r outputs, where each output G_a(z) is a d-dimensional vector. Add trainable structure constants C in R^(r x r x r). For each minibatch, evaluate all generators, compute Jacobian-vector products J_Gb G_a and J_Ga G_b with automatic differentiation, form every bracket, and regress it onto the generator span using C. Add the closure, Jacobi, antisymmetry, and rank terms to the ordinary reconstruction, prediction, or classification loss. Use exact Jacobian-vector products for small MLPs; for larger networks use a random subset of generator pairs or Hutchinson directional estimates. The Lie-bracket constraints come from the paper; the latent representation, task loss, and covariance penalty are adaptations. During the first warm-up phase, normalize each generator and stop gradients from the algebra losses into the encoder to avoid trivial zero fields. First test an autoencoder or latent predictor on trajectories from a 3D isotropic Ornstein-Uhlenbeck process, whose rotational symmetry has three nontrivial generators. Compare an unconstrained model with the regularized model and measure reconstruction, rotated-test prediction, closure residual, Jacobi residual, and singular values of the generator covariance. The prediction is that normalized closure residual falls below 10^-2 while three singular values remain nonzero; increasing closure weight past a critical value should cause a measurable rank-collapse transition. ## Verification - Status: mech_ok_no_baseline - Mechanism evidence: yes - Mechanism confirmed: yes - Practical verdict: no_effect - Verdict: Built a readable linear-generator MVP with exact Lie brackets, fitted structure constants, closure/Jacobi/antisymmetry checks, covariance singular values, and a regularized fitting sweep. The mechanism manifested: exact SO(3) closure/Jacobi were below 6e-33, perturbation closure MSE followed the predicted quadratic scaling with log-log slope 1.9999, and a high regularization weight produced clear rank collapse. The regularizer did not improve the already exact toy task and hurt task fit at extreme weight, so this is evidence for the mechanism rather than a general performance win. ### Mechanism check - Verdict: Built a readable linear-generator MVP with exact Lie brackets, fitted structure constants, closure/Jacobi/antisymmetry checks, covariance singular values, and a regularized fitting sweep. The mechanism manifested: exact SO(3) closure/Jacobi were below 6e-33, perturbation closure MSE followed the predicted quadratic scaling with log-log slope 1.9999, and a high regularization weight produced clear rank collapse. The regularizer did not improve the already exact toy task and hurt task fit at extreme weight, so this is evidence for the mechanism rather than a general performance win. - Confidence: 8/10 - Limitations: This is a linear vector-field toy rather than the proposed encoder plus nonlinear MLP on OU trajectories. Direct vector-field Jacobi is an identity, so the diagnostic uses Jacobi residuals of fitted constant structure coefficients. No rotated-test prediction, reconstruction model, CUDA path, or larger-network automatic differentiation benchmark was tested. ### Practical benchmark - Paired seeds: 8 - Baseline mean: 0 - Idea mean: 0 - p-value: 1 - Paired wins: 0/8 - Benchmark verdict: no measurable effect ## Artifacts - [bench_report.json](https://synthcore.org/code/545/bench_report.json) - [experiment.py](https://synthcore.org/code/545/experiment.py) - [report.md](https://synthcore.org/code/545/report.md) - [report_bench_2026-08-31T173233.md](https://synthcore.org/code/545/report_bench_2026-08-31T173233.md) - [results.json](https://synthcore.org/code/545/results.json) - [stage2_bench.py](https://synthcore.org/code/545/stage2_bench.py) - [Download all files as ZIP](https://synthcore.org/download/545) ## Disclaimer AI-generated research hypothesis, automatically tested. Not peer-reviewed.