# Gauge-Patched Local Experts

- ID: 1270
- Canonical URL: https://synthcore.org/idea/1270/gauge-patched-local-experts
- API JSON: https://synthcore.org/api/idea/1270.json
- API Markdown: https://synthcore.org/api/idea/1270.md
- Verification status: beats_baseline
- Source: [arXiv:2607.22236](https://arxiv.org/abs/2607.22236)
- Category: architecture
- Solves: accuracy, scalability, stability
- ML areas: graph-nn, moe-routing, embedding, regularization
- Math tags: lie-groups, geometry, representation-theory, analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Represent hidden features with several local orthogonal or unitary frames, and let group-valued transition maps align neighboring experts. Instead of forcing every expert to learn a globally coherent coordinate system, train local experts independently and impose a patching constraint that produces a globally consistent feature field. This is especially suitable for graph neural networks on data with heterogeneous local geometries or mixture-of-experts models whose experts specialize by region.

## Mathematical statement

The paper's patching result assumes local fields B_n and transition functions g_{n,m} on overlapping patches U_n, with max_{n,m}{|B_n|_{beta;U_n}+|g_{n,m}-1_G|_{infty;U_n intersection U_m}} <= delta_patch. Here beta is a Holder exponent, G is a compact connected Lie group, B_n is a local Lie-algebra-valued field, g_{n,m} is a group-valued transition map, and delta_patch is a smallness threshold depending on beta and G. It constructs a global field B and local gauge transformations q_n satisfying B=B_n^{q_n} on U_n and q_n=q_m g_{n,m} on overlaps. For neural vector features, use an orthogonal representation rho:G -> O(d) and define the transformed feature by B_n^{q_n}(x)=rho(q_n(x))B_n(x). Lemma 5.6 gives a stable transition extension: for disjoint V_1,V_2 contained in V, one can construct hat g equal to the identity on V_1 and g on V_2 with Holder norm bounded by d(V_1,V_2)^(-beta)|g-1_G|_{C^beta(V)}. The transferable property is controlled local coordinate change and composability of neighboring frames.

## Key formulas

- $$\max_{\mathbf n,\mathbf m}\left\{|B_{\mathbf n}|_{\beta;U_{\mathbf n}}+|g_{\mathbf n,\mathbf m}-1_G|_{\infty;U_{\mathbf n}\cap U_{\mathbf m}}\right\}\leq\delta_{\mathrm{patch}}.$$
- $$B=B_{\mathbf n}^{q_{\mathbf n}}\quad\text{on }U_{\mathbf n},\qquad q_{\mathbf n}=q_{\mathbf m}g_{\mathbf n,\mathbf m}\quad\text{on }U_{\mathbf n}\cap U_{\mathbf m}.$$
- $$|\hat g-1_G|_{C^\beta(V)}\lesssim d(V_1,V_2)^{-\beta}|g-1_G|_{C^\beta(V)}.$$
- $$\mathcal L_{\mathrm{patch}}=\sum_{(n,m)}\mathbb E_{x\in U_n\cap U_m}\left[\left\|h_n(x)-\rho(g_{n,m}(x))h_m(x)\right\|_2^2\right]+\lambda\sum_{(n,m)}\|g_{n,m}(x)-I\|_F^2.$$

## Implementation notes

(1) Integration point: insert this into a graph-NN or MoE hidden layer. Partition inputs into K overlapping patches or route tokens to K local experts. Expert n outputs h_n(x) in R^d. Parameterize every transition as g_nm(x)=exp(S_nm(x)), where S_nm is skew-symmetric, so g_nm is orthogonal and preserves feature norm. A transition MLP receives the shared input embedding and predicts the entries of S_nm.

(2) Pseudocode:
```
for batch x:
    for patch n:
        h[n] = expert_n(x[n])
    for overlap (n,m):
        S = skew(transition_nm(x_overlap))
        g = matrix_exp(S)
        L_patch += mean(||h[n]-g @ h[m]||_2^2)
        L_patch += lambda_id * mean(||g-I||_F^2)
    for triple (n,m,k):
        L_patch += lambda_cocycle * mean(||g_nm @ g_mk-g_nk||_F^2)
    L = task_loss + lambda_patch * L_patch
    update(parameters)
```
For a global readout, align each h_n to a reference patch using transition products along a spanning tree. (3) The paper contributes the small-overlap condition and cocycle equations. Estimate local Holder size by max_{i,j}||h(x_i)-h(x_j)||/||x_i-x_j||^beta, and monitor whether transition magnitudes and patch residuals remain below a chosen empirical delta_patch. (4) First experiment: train a four-layer GCN on a synthetic planar point-cloud classification task with random independent rotations applied to each spatial patch. Compare a shared GCN, ordinary MoE, and gauge-patched MoE at equal parameter count and FLOPs. Success means higher accuracy under unseen local rotations, lower prediction variance across rotations, lower overlap disagreement, and stable optimization as the number of experts grows.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a compact Gauge-Patched Local Experts MVP with orthogonal 2D transitions, Holder regularity checks, small-angle patch residual analysis, and a matched two-expert optimization sweep. All three quantitative predictions were confirmed: norm error was 2.2e-16, the observed Holder log-log slope was -0.7496 versus predicted -0.75, and the residual slope was 1.9995 versus predicted 2.0. The patch penalty reduced overlap disagreement from 0.7203 to 0.0800 at lambda=1 and to 0.00163 at lambda=10, but increased task MSE, so this demonstrates the mechanism rather than an overall accuracy win.

### Mechanism check

- Verdict: Built a compact Gauge-Patched Local Experts MVP with orthogonal 2D transitions, Holder regularity checks, small-angle patch residual analysis, and a matched two-expert optimization sweep. All three quantitative predictions were confirmed: norm error was 2.2e-16, the observed Holder log-log slope was -0.7496 versus predicted -0.75, and the residual slope was 1.9995 versus predicted 2.0. The patch penalty reduced overlap disagreement from 0.7203 to 0.0800 at lambda=1 and to 0.00163 at lambda=10, but increased task MSE, so this demonstrates the mechanism rather than an overall accuracy win.
- Confidence: 9/10
- Limitations: Only a 2D analytic toy and two linear local experts were tested; no graph neural network, learned transition MLP, matrix exponential parameterization beyond the SO(2) analytic equivalent, unseen-rotation classification, cocycle triples, GPU scaling, or equal-FLOP multi-expert benchmark was evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_gauge_dynamics.py](https://synthcore.org/code/508/bench_gauge_dynamics.py)
- [bench_report.json](https://synthcore.org/code/508/bench_report.json)
- [gauge_patched_experts.py](https://synthcore.org/code/508/gauge_patched_experts.py)
- [report.md](https://synthcore.org/code/508/report.md)
- [report_bench_2026-08-31T165041.md](https://synthcore.org/code/508/report_bench_2026-08-31T165041.md)
- [rerun_output.json](https://synthcore.org/code/508/rerun_output.json)
- [results.json](https://synthcore.org/code/508/results.json)
- [Download all files as ZIP](https://synthcore.org/download/508)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.