# Weyl-conjugation layer regularization

- ID: 3046
- Canonical URL: https://synthcore.org/idea/3046/weyl-conjugation-layer-regularization
- API JSON: https://synthcore.org/api/idea/3046.json
- API Markdown: https://synthcore.org/api/idea/3046.md
- Verification status: unverified
- Source: [arXiv:2609.02261](https://arxiv.org/abs/2609.02261)
- Category: regularization
- Solves: stability, generalization
- ML areas: regularization, cnn, transformer, training-dynamics
- Math tags: probability, lie-groups, harmonic-analysis, functional-analysis
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a learned linear operator by Monte Carlo averages of unitary-conjugated copies, where the conjugators are discrete time-frequency shifts sampled from a narrow phase-space distribution. The conjugation preserves the singular spectrum of each sampled operator, while the expectation penalizes sensitivity to small translations and modulations without directly shrinking the operator norm.

## Mathematical statement

The paper defines a random unitary operator from a phase-space process by $V(t,\omega)=W_h(X(t,\omega))$, where $X(t,\omega)\in\mathbb{R}^{2n}$ is a random phase-space point, $h>0$ is the quantization scale, and $W_h$ is the Weyl translation. Given a base operator $A_0$, the random conjugate is $A(t,\omega)=V(t,\omega)^\star A_0V(t,\omega)$ and the associated quantization is its expectation, ${\rm Op}_{t}^{\nu_t}(F)=\int_\Omega A(t,\omega)d\mu_\Omega(\omega)$. For $X=(q,p)$ with position shift $q\in\mathbb{R}^n$ and modulation $p\in\mathbb{R}^n$, use $(W_h(q,p)z)(x)=\exp(i p\cdot(x-q/2)/h)z(x-q)$. Because $W_h(q,p)$ is unitary, every sampled operator satisfies $\|W_h(q,p)^\star A_0W_h(q,p)\|_2=\|A_0\|_2$ and has the same singular values as $A_0$. The ML adaptation estimates the expectation with $K$ samples and trains either the averaged operator or one fresh conjugate per minibatch.

## Key formulas

- $$V(t,\omega)=W_h(X(t,\omega))$$
- $$A(t,\omega)=V(t,\omega)^{\star}A_0V(t,\omega)$$
- $${\rm Op}_{t}^{\nu_t}(F)=\int_{\Omega}A(t,\omega)d\mu_\Omega(\omega)$$
- $$\widetilde A_{K}(z)=\frac{1}{K}\sum_{k=1}^{K}W_h(q_k,p_k)^{\star}A_0W_h(q_k,p_k)z,\qquad (W_h(q,p)z)(x)=e^{i p\cdot(x-q/2)/h}z(x-q)$$

## Implementation notes

Integrate this at a linear feature operator, not at nonlinear activations. The cheapest first target is a 2D convolution or a ViT patch-mixing/projection layer whose feature map has spatial dimensions H by W and channel dimension C. Store the trainable base operator A0. At every training step sample K phase-space perturbations (qk, pk), with qk drawn from a zero-centered discrete Gaussian over shifts and pk drawn from a zero-centered discrete Gaussian over modulation frequencies. For each sample construct Wk by circularly shifting the spatial feature map by qk and multiplying each spatial location by exp(i pk dot x / h); use real-valued features by implementing the complex operation as a two-channel real rotation, or initially use only signed cosine modulation. Compute yk = Wk inverse(A0(Wk(x))) and return y = (1/K) sum yk. Pseudocode: x -> for k: xk=W(qk,pk)x; uk=A0(xk); yk=W(qk,pk)^T uk; output mean(yk). The exact imported mathematics is unitary conjugation and expectation; q and p distributions, K, and h are engineering choices. Estimate no spectral quantities: verify numerically that each Wk preserves the feature norm and that sampled Ak has the same singular values as A0. Begin with CIFAR-10, a small ResNet-18, and a baseline with identical parameter count and FLOPs. Compare K=1 stochastic training, K=2 averaging, and no conjugation at equal training budgets. Success means improved corruption and translation robustness and validation accuracy, or reduced train-test gap, without exploding activation norms; report the extra wall-clock cost and test whether narrow noise beats ordinary additive feature noise.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.