# Invariant-Lattice Adapter

- ID: 2650
- Canonical URL: https://synthcore.org/idea/2650/invariant-lattice-adapter
- API JSON: https://synthcore.org/api/idea/2650.json
- API Markdown: https://synthcore.org/api/idea/2650.md
- Verification status: unverified
- Source: [arXiv:2608.21238](https://arxiv.org/abs/2608.21238)
- Category: regularization
- Solves: memory, generalization, accuracy
- ML areas: fine-tuning, transformer, memory, regularization
- Math tags: algebra, geometry, representation-theory, linear-algebra
- Ratings: usefulness 5/10; difficulty 4/10; novelty 8/10

## Idea description

Restrict a fine-tuning adapter or output head to the subspace invariant under a prescribed monodromy, analogous to the paper's unbroken flavor lattice. This removes update directions intentionally changed by the domain-loop transformation, producing a parameter-efficient adapter with an explicit algebraic constraint.

## Mathematical statement

For the n=4 construction, the paper states that the monodromy acts on simple roots by \(\alpha_1\mapsto\alpha_1-4E\), \(\alpha_3\mapsto\alpha_3+2E\), and \(\alpha_i\mapsto\alpha_i\) for \(i=2,4,5,6,7,8\). The invariant root sublattice is \(\Lambda_{n=4}=\alpha_1^\perp\subset\Lambda_{E_8}\), explicitly generated by \(v_1=\alpha_1+\alpha_2+2\alpha_3+2\alpha_4+\alpha_5\) and \(v_2=\alpha_2,v_3=\alpha_4,v_4=\alpha_5,v_5=\alpha_6,v_6=\alpha_7,v_7=\alpha_8\); the paper identifies this lattice with \(-E_7\). Let \(V=[v_1,\ldots,v_7]\) be the d-by-7 matrix of these generators in feature coordinates and let G be the feature-space inner-product matrix. The projector onto the invariant subspace is \(P=V(V^TGV)^{-1}V^TG\), satisfying \(P^2=P\). A LoRA update can be constrained as \(\Delta W=AP\), or an unconstrained update can be projected after every optimizer step.

## Key formulas

- $$\mathcal{M}[4\alpha_1]:\quad \alpha_1\mapsto\alpha_1-4E,\qquad \alpha_3\mapsto\alpha_3+2E,\qquad \alpha_i\mapsto\alpha_i\quad(i=2,4,5,6,7,8)$$
- $$\Lambda_{n=4}=\langle\alpha_1+\alpha_2+2\alpha_3+2\alpha_4+\alpha_5,\alpha_2,\alpha_4,\alpha_5,\alpha_6,\alpha_7,\alpha_8\rangle_{\mathbb Z}=\alpha_1^\perp$$
- $$P=V(V^TGV)^{-1}V^TG,\qquad P^2=P$$
- $$\Delta W=AP\qquad\text{or}\qquad\Delta W\leftarrow\Delta W P$$

## Implementation notes

Apply this to LoRA fine-tuning in a Transformer attention projection, preferably the query, key, or value matrix, where parameter and update reductions are easy to inspect. Select d feature coordinates and construct seven basis vectors from the paper's invariant generators. For an MVP, embed the symbolic roots α_i as standard coordinate vectors in R^8, giving v1=(1,1,2,2,1,0,0,0), v2=(0,1,0,0,0,0,0,0), v3=(0,0,0,1,0,0,0,0), v4=(0,0,0,0,1,0,0,0), v5=(0,0,0,0,0,1,0,0), v6=(0,0,0,0,0,0,1,0), and v7=(0,0,0,0,0,0,0,1). Use G=I initially, orthonormalize V numerically, and compute P=V(V^TV)^(-1)V^T once. Replace ordinary LoRA ΔW=BA with ΔW=BA P, or parameterize B'=BP so every update is invariant. During training, optionally add λ||ΔW−TΔW T^(-1)||_F^2 when an explicit monodromy T is available, but the hard projection is the primary MVP. The paper supplies the invariant generators and subspace dimension; empirical choices are adapter rank and insertion point. Compare full LoRA, ordinary LoRA at equal parameter count, and projected LoRA on a small ViT or language model over several classification or instruction-tuning tasks. Record trainable parameters, peak memory, validation accuracy, domain-shift accuracy, and cross-task forgetting. Success means matching ordinary LoRA with fewer parameters or reducing forgetting at equal parameter count.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.