# Commutator-Regularized Switched SSM

- ID: 449
- Canonical URL: https://synthcore.org/idea/449/commutator-regularized-switched-ssm
- API JSON: https://synthcore.org/api/idea/449.json
- API Markdown: https://synthcore.org/api/idea/449.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.05829](https://arxiv.org/abs/2607.05829)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: ssm, training-dynamics, regularization
- Math tags: dynamical-systems, linear-algebra, control-theory, optimization
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Build a state-space layer whose latent dynamics use a fixed cyclic schedule of learned generators instead of a single generator. Penalize pairwise commutator norms so that the true ordered cycle remains close to the averaged flow, while periodically checking a quadratic Lyapunov contraction condition on the exact cycle transition.

## Mathematical statement

The paper studies \(\dot{x}(t)=A_{\sigma(t)}x(t)\), with mode \(i\) active for \(\alpha_i\tau\) during period \(\tau\), where \(\alpha\in\Delta^{m-1}=\{\alpha_i\ge 0,\sum_i\alpha_i=1\}\). Define \(B_i=\alpha_i\tau A_i\), the averaged generator \(\bar A=\sum_i\alpha_iA_i\), the averaged propagator \(\bar\Phi=\exp(\tau\bar A)=\exp(\sum_iB_i)\), and the ordered switched monodromy \(\Phi=\exp(B_m)\cdots\exp(B_1)\). The relevant noncommutativity measure is the matrix commutator \([B_i,B_j]=B_iB_j-B_jB_i\). The BCH expansion gives the leading mismatch through pairwise commutators: \(\prod_i\exp(B_i)=\exp(\sum_iB_i+\frac12\sum_{i>j}[B_i,B_j]+O(\max_i\|B_i\|^3))\). A practical norm surrogate is \(R_{\mathrm{comm}}=\sum_{i>j}\|[B_i,B_j]\|_F^2\), which vanishes for mutually commuting generators. Stability is checked using a quadratic Lyapunov matrix \(P\succ0\): for contraction factor \(0<\rho<1\), require \(\Phi(\alpha)^TP\Phi(\alpha)-\rho^2P\preceq0\) for sampled duty vectors \(\alpha\). Here \(A_i\in\mathbb R^{d\times d}\) are learned latent generators, \(d\) is state width, \(\tau\) is the cycle duration, \(\alpha_i\) are duty fractions, and \(P\) defines the contraction metric.

## Key formulas

- $$\dot{x}(t)=A_{\sigma(t)}x(t),\qquad \alpha\in\Delta^{m-1}:=\left\{\alpha\in\mathbb{R}^m_{\ge 0}:\sum_{i=1}^m\alpha_i=1\right\}.$$
- $$\Phi(\alpha)=e^{B_m}\cdots e^{B_1},\qquad B_i=\alpha_i\tau A_i,\qquad \bar\Phi(\alpha)=e^{\sum_{i=1}^mB_i}=e^{\tau\bar A},\quad \bar A=\sum_{i=1}^m\alpha_iA_i.$$
- $$\prod_{i=1}^{m}e^{B_i}=\exp\left(\sum_{i=1}^{m}B_i+\frac12\sum_{i>j}[B_i,B_j]+O\left(\max_i\|B_i\|^3\right)\right),\qquad [B_i,B_j]=B_iB_j-B_jB_i.$$
- $$\mathcal{L}=\mathcal{L}_{\mathrm{task}}+\lambda_{c}\sum_{i>j}\|[B_i,B_j]\|_{F}^{2}+\lambda_{s}\sum_{\alpha\in\mathcal{A}}\left\|\Phi(\alpha)^TP\Phi(\alpha)-\rho^2P\right\|_{+}^{2},\qquad P\succ0,\ 0<\rho<1.$$

## Implementation notes

Integrate this as a replacement for the recurrent or state-update core of a small SSM or gated RNN. Use latent state \(h_t\in\mathbb R^d\), input projection \(U\), output projection \(V\), and \(m\) learned generators \(A_i\). For each cycle, choose duty fractions \(\alpha\) and perform substeps in the fixed order 1 through \(m\): `h = E[i] @ h + input_substep[i]`, where `B[i] = alpha[i] * tau * A[i]` and `E[i] = matrix_exp(B[i])`; emit the output after the complete cycle. Parameterize \(A_i\) with a bounded spectral scale, for example \(A_i=S_i-L_iL_i^T-\epsilon I\), but do not assume individual-mode stability is sufficient because the ordered product is the actual transition. During training, compute `comm = sum(frob(B[i] @ B[j] - B[j] @ B[i])**2 for i>j)` and add `lambda_c * comm` to the task loss. Every few hundred optimizer steps, estimate a Lyapunov metric by solving an SDP over a small duty set \(\mathcal A\): minimize \(-\log\det(P)\) subject to \(P\succeq\epsilon I\) and \(\Phi(\alpha)^T P\Phi(\alpha)\preceq\rho^2P\) for each sampled \(\alpha\). For a cheaper MVP, use diagonal \(P\) and optimize its positive diagonal entries by gradient descent. The paper-derived quantities are the monodromy, averaged generator, and commutator expansion; the stability margin is estimated empirically as the maximum generalized eigenvalue of \((\Phi^TP\Phi,P)\) over held-out duty vectors. First test a width-32 or width-64 switched SSM on sequential or permuted-MNIST against a single-generator SSM and an unregularized switched SSM with matched parameter count and FLOPs. Sweep cycle period, duty fractions, and mode order. Success means fewer exploding states, a lower maximum Lyapunov gain, and equal or better validation accuracy; a particularly clear win is stable training on schedules where the unregularized switched model diverges, without a significant increase in per-token latency.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a switched matrix-exponential SSM MVP with fixed mode ordering, commutator regularization, BCH verification, and cycle spectral diagnostics. The corrected BCH experiment showed that including the leading commutator term reduced mismatch scaling toward cubic (log-log slope 2.69 versus 1.49 without it), while direct optimization greatly reduced commutator error and ordered-versus-averaged mismatch without harming task fit. In the small sequence regression, the regularized switched model achieved the best validation MSE, but this is only a promising toy signal rather than evidence of a general win.

### Mechanism check

- Verdict: Built a switched matrix-exponential SSM MVP with fixed mode ordering, commutator regularization, BCH verification, and cycle spectral diagnostics. The corrected BCH experiment showed that including the leading commutator term reduced mismatch scaling toward cubic (log-log slope 2.69 versus 1.49 without it), while direct optimization greatly reduced commutator error and ordered-versus-averaged mismatch without harming task fit. In the small sequence regression, the regularized switched model achieved the best validation MSE, but this is only a promising toy signal rather than evidence of a general win.
- Confidence: 7/10
- Limitations: The learning experiment is a tiny synthetic scalar sequence regression with one random seed and only two modes; no MNIST, language modeling, FLOP/latency benchmark, duty/order sweep, or learned quadratic Lyapunov P/SDP check was performed. The BCH cubic slope is approximate because the tested scale range includes non-asymptotic step sizes.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.6188
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_commutator_dynamics.py](https://synthcore.org/code/1306/bench_commutator_dynamics.py)
- [bench_experiment.py](https://synthcore.org/code/1306/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1306/bench_report.json)
- [commutator_ssm_mvp.py](https://synthcore.org/code/1306/commutator_ssm_mvp.py)
- [report.md](https://synthcore.org/code/1306/report.md)
- [report_bench_2026-09-04T145942.md](https://synthcore.org/code/1306/report_bench_2026-09-04T145942.md)
- [results.json](https://synthcore.org/code/1306/results.json)
- [stage2_bench.py](https://synthcore.org/code/1306/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/1306)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.