# Commutant-Gap Controlled Stochastic Training

- ID: 2118
- Canonical URL: https://synthcore.org/idea/2118/commutant-gap-controlled-stochastic-training
- API JSON: https://synthcore.org/api/idea/2118.json
- API Markdown: https://synthcore.org/api/idea/2118.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.11297](https://arxiv.org/abs/2608.11297)
- Category: dynamics
- Solves: stability, generalization
- ML areas: optimizer, training-dynamics, rnn, ssm
- Math tags: dynamical-systems, control-theory, spectral-theory, linear-algebra, statistical-mechanics
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Replace unconstrained parameter or hidden-state noise by Brownian perturbations generated by symmetry-preserving directions, then monitor the effective replica generator on k copies of the hidden representation. The smallest nonzero eigenvalue of this generator is a measurable relaxation gap: maintain it above a target to avoid frozen symmetry sectors, while reducing noise when the gap collapses. This transfers the paper's symmetry-controlled low-energy geometry into an optimizer and recurrent-network stability controller.

## Mathematical statement

The paper models noisy dynamics as H(t)=sum_alpha J_alpha(t)h_alpha, with independent zero-mean white noises satisfying E[J_alpha(t)J_beta(t')]=2 kappa delta_alpha,beta delta(t-t'). Averaging gives Ubar(t)=exp(-kappa P t), where P=sum_alpha h_alpha squared is positive semidefinite. For neural-network hidden states z in R^d with continuous symmetry generators T_a, define the k-replica generator T_a^(k)=sum_r I tensor ... tensor T_a tensor ... tensor I. The k-commutant is C_k={X:[X,T_a^(k)]=0 for every a}. If P_k is the empirical replica noise generator and Pi_k projects onto C_k, define Delta_k=lambda_min^+(P_k restricted to C_k perpendicular). The non-commutant component then obeys the contraction bound ||exp(-kappa P_k t)(I-Pi_k)||_2 <= exp(-kappa Delta_k t). Thus Delta_k predicts the slowest symmetry-compatible relaxation rate.

## Key formulas

- $$H(t)=\sum_{\alpha}J_{\alpha}(t)h_{\alpha},\qquad \mathbb{E}[J_{\alpha}(t)J_{\beta}(t')]=2\kappa\,\delta_{\alpha\beta}\delta(t-t')$$
- $$\overline{U(t)}=e^{-\kappa P t},\qquad P=\sum_{\alpha}h_{\alpha}^{2}\succeq 0$$
- $$\mathcal{C}_k=\{X:[X,T_a^{(k)}]=0\ \forall a\},\qquad T_a^{(k)}=\sum_{r=1}^{k}I^{\otimes(r-1)}\otimes T_a\otimes I^{\otimes(k-r)}$$
- $$\Delta_k=\lambda_{\min}^{+}\!\left(P_k\big|_{\mathcal{C}_k^{\perp}}\right),\qquad \|e^{-\kappa P_k t}(I-\Pi_k)\|_2\le e^{-\kappa\Delta_k t}$$

## Implementation notes

Use a residual RNN or linear state-space model with paired hidden coordinates and a known SO(2) rotation symmetry, for example z_{t+1}=z_t+eta f_theta(z_t,x_t). Parameterize stochastic perturbations as delta f=sum_alpha xi_alpha h_alpha(z), where each h_alpha commutes with the rotation generator T. Maintain k independent hidden replicas during training. At every M optimizer steps, estimate the covariance of replica feature increments and assemble an empirical positive-semidefinite generator P_k. Compute a null-space basis for matrices X satisfying [X,T_a^(k)]=0 using SVD, project P_k onto the orthogonal complement, and estimate Delta_k by the smallest positive eigenvalue. Sample xi_alpha from N(0,2 kappa Delta_t), and update theta with the ordinary gradient plus this symmetry-preserving perturbation. If Delta_k falls below a target, halve kappa and optionally reduce eta; otherwise slowly restore kappa. The paper supplies the Brownian averaging and gap mechanism; P_k and Delta_k are estimated from minibatch trajectories. First test a 32-unit equivariant residual RNN on sequential MNIST and a rotating-signal prediction task against SGD and fixed-noise SGD. The predicted signature is perturbation decay exp(-kappa Delta_k t), with fitted decay rate within 20% of kappa Delta_k; long-horizon error should increase sharply as Delta_k approaches zero.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a reproducible commutant-gap experiment with exact SO(2) two-replica commutant projection, PSD generator construction, gap scaling sweeps, and a small symmetry-preserving noisy regression controller. The mechanism manifested: fitted decay rates matched kappa*Lambda within 2.1e-15 relative error, half-lives matched within 1.0%, and the Lambda=0 case showed no decay while commutant directions remained unchanged. In toy training, gap control reduced final loss versus fixed noise (0.0078646 vs 0.015729) but did not beat noiseless SGD (1.46e-31), so this validates the mechanism rather than demonstrating a general optimization win.

### Mechanism check

- Verdict: Built a reproducible commutant-gap experiment with exact SO(2) two-replica commutant projection, PSD generator construction, gap scaling sweeps, and a small symmetry-preserving noisy regression controller. The mechanism manifested: fitted decay rates matched kappa*Lambda within 2.1e-15 relative error, half-lives matched within 1.0%, and the Lambda=0 case showed no decay while commutant directions remained unchanged. In toy training, gap control reduced final loss versus fixed noise (0.0078646 vs 0.015729) but did not beat noiseless SGD (1.46e-31), so this validates the mechanism rather than demonstrating a general optimization win.
- Confidence: 9/10
- Limitations: Only a low-dimensional analytic SO(2) toy and equivariant linear regression were tested; no sequential MNIST, nonlinear 32-unit RNN, minibatch empirical replica-generator estimation, CUDA run, or generalization evaluation was performed. The training controller uses a prescribed synthetic gap schedule rather than estimating Delta_k from trajectories.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.93175
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_commutant_gap.py](https://synthcore.org/code/854/bench_commutant_gap.py)
- [bench_report.json](https://synthcore.org/code/854/bench_report.json)
- [commutant_gap_experiment.py](https://synthcore.org/code/854/commutant_gap_experiment.py)
- [report.md](https://synthcore.org/code/854/report.md)
- [report_bench_2026-09-01T125605.md](https://synthcore.org/code/854/report_bench_2026-09-01T125605.md)
- [results.json](https://synthcore.org/code/854/results.json)
- [Download all files as ZIP](https://synthcore.org/download/854)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.