# Controllability-Guided Low-Mode Adapter

- ID: 446
- Canonical URL: https://synthcore.org/idea/446/controllability-guided-low-mode-adapter
- API JSON: https://synthcore.org/api/idea/446.json
- API Markdown: https://synthcore.org/api/idea/446.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.05687](https://arxiv.org/abs/2607.05687)
- Category: architecture
- Solves: memory, scalability, accuracy
- ML areas: fine-tuning, ssm, embedding, inference-speedup
- Math tags: differential-geometry, lie-groups, harmonic-analysis, dynamical-systems
- Ratings: usefulness 7/10; difficulty 6/10; novelty 9/10

## Idea description

Use a small number of learned low-mode controls and a fixed bank of Lie words to generate structured high-mode updates. This gives a parameter-efficient adapter for spectral operators or sequence models: the trainable degrees of freedom live only in the low modes, while commutator compositions provide deterministic propagation paths to larger offsets. The design is especially suitable for fine-tuning a pretrained Fourier or state-space model under a strict parameter budget.

## Mathematical statement

The paper represents the state as Fourier coefficients $\hat{\mathfrak m}=(\hat{\mathbf m}_{0}^{\top},\hat{\mathbf m}_{1}^{\top},\ldots)^{\top}$ and observes that shifted fields act as $G_{m,k}(\hat{\mathfrak m})_{i}=\mathbf g^{\ast}_{m}(\hat{\mathbf m}_{i+k})$. The central constructive fact is that commutators of shifted fields move to the summed index: $[G_{m,k},G_{n,l}]_{i}=[\mathbf g^{\ast}_{m},\mathbf g^{\ast}_{n}](\hat{\mathbf m}_{i+k+l})$. Repeated brackets therefore generate offsets $k_{1}+\cdots+k_{q}$ while retaining tangent directions. For a Lie word $w=((m_{1},k_{1}),\ldots,(m_{q},k_{q}))$, define recursively $V_{w}= [G_{m_{1},k_{1}},V_{w^{-}}]$ and total offset $s(w)=\sum_{j=1}^{q}k_{j}$. The associated finite-difference flow word has leading term $\varepsilon^{q}V_{w}$, so a small low-mode control can influence mode $i+s(w)$ through a depth-$q$ composition. The sphere constraint is preserved by each primitive rotational flow because its vector field is tangent: $\mathbf x\cdot\mathbf g^{\ast}_{m}(\mathbf x)=0$.

## Key formulas

- $$G_{m,k}(\hat{\mathfrak m})=\big(\mathbf g^{\ast}_{m}(\hat{\mathbf m}_{i+k})\big)_{i=0}^{\infty},\qquad [G_{m,k},G_{n,l}](\hat{\mathfrak m})=\big([\mathbf g^{\ast}_{m},\mathbf g^{\ast}_{n}](\hat{\mathbf m}_{i+k+l})\big)_{i=0}^{\infty}.$$
- $$V_{((m_{1},k_{1}),\ldots,(m_{q},k_{q}))}:=[G_{m_{1},k_{1}},V_{((m_{2},k_{2}),\ldots,(m_{q},k_{q}))}],\qquad s(w)=\sum_{j=1}^{q}k_{j}.$$
- $$\Phi_{w,\varepsilon}(X)=X+\varepsilon^{q}V_{w}(X)+O(\varepsilon^{q+1}),\qquad \|X_i\|_{2}=1\ \Longrightarrow\ \|\Phi_{w,\varepsilon}(X)_i\|_{2}=1\ \text{for exact primitive flows}.$$
- $$X'_{i}=X_{i}+\sum_{w\in\mathcal W:\,s(w)=r}\alpha_{w}\,\varepsilon^{|w|}V_{w}(X)_{i},$$

## Implementation notes

Integrate this as a parameter-efficient fine-tuning adapter after the spectral projection or state update of a pretrained Fourier neural operator, S4-like state-space model, or periodic sequence encoder. Let the hidden tensor be `X[B,N,3]`; reserve the first `K=4` positions or Fourier modes as low-mode control locations. Define a fixed dictionary of words of lengths one and two, for example `(1,1)`, `(2,1)`, `(1,2)`, `(2,2)`, and the bracket words `[(1,1),(2,1)]`, `[(1,2),(2,2)]`, with target offsets 2, 3, and 4. Compute primitive shifted fields with gather operations. For a length-two word, compute `V = J_g1(g2(X_shifted)) - J_g2(g1(X_shifted))`; because the fields are linear, this can be hard-coded as the corresponding cross-product rotation rather than using autograd Jacobians. Add only trainable scalar coefficients `alpha_w` and, optionally, a trainable `K x 3` control tensor; freeze the backbone. Use `X_new = normalize(X + sum(alpha_w * eps**len(w) * V_w(X)))`, or implement each word through exact small-angle 3D rotation matrices to avoid normalization artifacts. The paper supplies the offset-addition rule and order scaling; coefficient magnitudes, word selection, and `eps` are empirical. Begin with a pretrained 2D Fourier classifier on CIFAR-10 or a synthetic band-limited-to-high-frequency regression task. Compare a full-rank spectral fine-tuning head and LoRA with the same trainable parameter count. The target signal is equal validation accuracy with 10–50x fewer trainable parameters, plus improved extrapolation to inputs containing frequencies above those seen during training. Measure per-step memory and the norm error `max_i | ||X_i||_2-1 |`.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a PyTorch Lie-mode adapter with fixed skew rotations, shifted fields, two hard-coded commutators, six trainable scalar coefficients, optional low-mode controls, and residual normalization. The toy checks confirmed all three predictions: summed offset k+l=3 exactly, epsilon-scaling remainder slope 1.004 versus predicted 1, and commutator magnitude proportional to sin(theta) with zero at aligned axes. In the matched synthetic regression, the commutator adapter achieved test MSE 0.00289 versus 56.08 for a one-parameter primitive-shift baseline, demonstrating the mechanism but not yet a real fine-tuning win.

### Mechanism check

- Verdict: Built a PyTorch Lie-mode adapter with fixed skew rotations, shifted fields, two hard-coded commutators, six trainable scalar coefficients, optional low-mode controls, and residual normalization. The toy checks confirmed all three predictions: summed offset k+l=3 exactly, epsilon-scaling remainder slope 1.004 versus predicted 1, and commutator magnitude proportional to sin(theta) with zero at aligned axes. In the matched synthetic regression, the commutator adapter achieved test MSE 0.00289 versus 56.08 for a one-parameter primitive-shift baseline, demonstrating the mechanism but not yet a real fine-tuning win.
- Confidence: 9/10
- Limitations: No CIFAR, Fourier neural operator, state-space model, pretrained-backbone fine-tuning, FLOP or memory benchmark, or multi-seed study was tested. The adapter uses residual normalization rather than composing exact primitive rotations, and the secondary regression is synthetic and intentionally generated from the commutator feature.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [adapter.py](https://synthcore.org/code/228/adapter.py)
- [bench_report.json](https://synthcore.org/code/228/bench_report.json)
- [bench_run.py](https://synthcore.org/code/228/bench_run.py)
- [report.md](https://synthcore.org/code/228/report.md)
- [report_bench_2026-08-30T211058.md](https://synthcore.org/code/228/report_bench_2026-08-30T211058.md)
- [results.json](https://synthcore.org/code/228/results.json)
- [test_adapter.py](https://synthcore.org/code/228/test_adapter.py)
- [verify_adapter.py](https://synthcore.org/code/228/verify_adapter.py)
- [Download all files as ZIP](https://synthcore.org/download/228)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.