# Generator-Flow Equivariance Training

- ID: 1677
- Canonical URL: https://synthcore.org/idea/1677/generator-flow-equivariance-training
- API JSON: https://synthcore.org/api/idea/1677.json
- API Markdown: https://synthcore.org/api/idea/1677.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.01582](https://arxiv.org/abs/2608.01582)
- Category: architecture
- Solves: generalization, sample-efficiency, accuracy
- ML areas: world-model, rnn, ssm, regularization, training-dynamics
- Math tags: dynamical-systems, differential-geometry, control-theory, numerical-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

## Mathematical statement

Let F_theta(z) be a latent neural dynamics field and G_a(z) a learned symmetry generator. Commuting flows require [G_a,F_theta](z)=J_{F_theta}(z)G_a(z)-J_{G_a}(z)F_theta(z)=0. For a discrete predictor P_theta, use the finite-flow condition P_theta(Phi_{a,epsilon}(z)) approximately equals Phi_{a,epsilon}(P_theta(z)), where Phi_{a,epsilon} is the time-epsilon flow generated by G_a. With Euler integration, Phi_{a,epsilon}(z)=z+epsilon G_a(z)+O(epsilon^2). If the first-order equivariance condition is satisfied, the finite-flow discrepancy is O(epsilon^2), and its squared norm is expected to scale as O(epsilon^4).

## Key formulas

- $$\Phi_{a,\epsilon}(z)=z+\epsilon G_a(z)+O(\epsilon^2)$$
- $$\mathcal{L}_{\mathrm{eq}}=\frac{1}{|\mathcal{B}|r|\mathcal{E}|}\sum_{z,a,\epsilon\in\mathcal{E}}\left\|P_\theta(\Phi_{a,\epsilon}(z))-\Phi_{a,\epsilon}(P_\theta(z))\right\|_2^2$$
- $$[G_a,F_\theta](z)=J_{F_\theta}(z)G_a(z)-J_{G_a}(z)F_\theta(z)$$
- $$\mathcal{L}_{\mathrm{inf}}=\frac{1}{|\mathcal{B}|r}\sum_{z,a}\|[G_a,F_\theta](z)\|_2^2,\qquad \mathcal{L}_{\mathrm{eq}}(\epsilon)=O(\epsilon^4)\ \text{in squared norm when first-order equivariance holds}$$

## Implementation notes

Insert the method into a latent predictor or world model. Encode x_t as z_t=E_theta(x_t), predict z_hat_(t+1)=P_theta(z_t), and obtain generators G_a from the learned Lie-algebra module. For each minibatch, select generator indices and amplitudes epsilon in {epsilon_0,2 epsilon_0}. Compute Phi(z)=z+epsilon G_a(z), evaluate P_theta(Phi(z)), evaluate Phi(P_theta(z)), and add their squared difference to the prediction loss. A more efficient infinitesimal implementation computes the commutator with two Jacobian-vector products and uses L_inf. Use several small epsilon values and stop gradients through G during early training so the system cannot satisfy equivariance by making generators zero. The learned-generator construction is taken from the paper; finite-flow augmentation and predictor commutation are the neural-network adaptation. First test on a latent predictor trained on rotated 2D damped-oscillator or isotropic Ornstein-Uhlenbeck trajectories, withholding several rotation angles during training. Compare ordinary prediction, random augmentation, and learned-flow equivariance. Plot equivariance error against epsilon on log-log axes. The quantitative prediction is that the squared error scales as epsilon^4 after first-order equivariance is learned, versus epsilon^2 for a non-equivariant baseline; rotated-test error should decrease when normalized commutator residual crosses below 10^-2.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a reproducible generator-flow equivariance MVP with analytic rotation generators, finite-flow discrepancy sweeps, and a CUDA-enabled latent predictor experiment. The core mechanism matched its quantitative predictions: squared discrepancy slopes were 4.0000 for an equivariant radial map and 2.0000 for a non-equivariant map across generator strengths, while lambda slopes were 4.0000 and 2.0000 respectively. In the learned predictor, rotated-test MSE improved from 0.01903 to 0.01461 (23.2%), although the measured finite-flow error at epsilon=0.05 increased, so the generalization signal is promising but not a uniform win.

### Mechanism check

- Verdict: Built a reproducible generator-flow equivariance MVP with analytic rotation generators, finite-flow discrepancy sweeps, and a CUDA-enabled latent predictor experiment. The core mechanism matched its quantitative predictions: squared discrepancy slopes were 4.0000 for an equivariant radial map and 2.0000 for a non-equivariant map across generator strengths, while lambda slopes were 4.0000 and 2.0000 respectively. In the learned predictor, rotated-test MSE improved from 0.01903 to 0.01461 (23.2%), although the measured finite-flow error at epsilon=0.05 increased, so the generalization signal is promising but not a uniform win.
- Confidence: 8/10
- Limitations: This is a small 2D controlled toy experiment using a known fixed rotation generator rather than a learned generator module. It did not test the paper's full SDE generator discovery, random augmentation baseline, multiple seeds, FLOPs-normalized comparisons, longer trajectories, or an infinitesimal commutator/JVP implementation.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.1336
- Paired wins: 2/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/546/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/546/bench_report.json)
- [experiment.py](https://synthcore.org/code/546/experiment.py)
- [report.md](https://synthcore.org/code/546/report.md)
- [report_bench_2026-08-31T180842.md](https://synthcore.org/code/546/report_bench_2026-08-31T180842.md)
- [results.json](https://synthcore.org/code/546/results.json)
- [Download all files as ZIP](https://synthcore.org/download/546)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.