Weyl-conjugation layer regularization
Implementation & benchmark of arXiv:2609.02261 — Quantization and process
Source paper: Quantization and process arXiv:2609.02261 ⓘ · analyzed Sep 3, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a learned linear operator by Monte Carlo averages of unitary-conjugated copies, where the conjugators are discrete time-frequency shifts sampled from a narrow phase-space distribution. The conjugation preserves the singular spectrum of each sampled operator, while the expectation penalizes sensitivity to small translations and modulations without directly shrinking the operator norm.
Formulas
Mathematical statement
The paper defines a random unitary operator from a phase-space process by $V(t,\omega)=W_h(X(t,\omega))$, where $X(t,\omega)\in\mathbb{R}^{2n}$ is a random phase-space point, $h>0$ is the quantization scale, and $W_h$ is the Weyl translation. Given a base operator $A_0$, the random conjugate is $A(t,\omega)=V(t,\omega)^\star A_0V(t,\omega)$ and the associated quantization is its expectation, ${\rm Op}_{t}^{\nu_t}(F)=\int_\Omega A(t,\omega)d\mu_\Omega(\omega)$. For $X=(q,p)$ with position shift $q\in\mathbb{R}^n$ and modulation $p\in\mathbb{R}^n$, use $(W_h(q,p)z)(x)=\exp(i p\cdot(x-q/2)/h)z(x-q)$. Because $W_h(q,p)$ is unitary, every sampled operator satisfies $\|W_h(q,p)^\star A_0W_h(q,p)\|_2=\|A_0\|_2$ and has the same singular values as $A_0$. The ML adaptation estimates the expectation with $K$ samples and trains either the averaged operator or one fresh conjugate per minibatch.
Implementation notes
Integrate this at a linear feature operator, not at nonlinear activations. The cheapest first target is a 2D convolution or a ViT patch-mixing/projection layer whose feature map has spatial dimensions H by W and channel dimension C. Store the trainable base operator A0. At every training step sample K phase-space perturbations (qk, pk), with qk drawn from a zero-centered discrete Gaussian over shifts and pk drawn from a zero-centered discrete Gaussian over modulation frequencies. For each sample construct Wk by circularly shifting the spatial feature map by qk and multiplying each spatial location by exp(i pk dot x / h); use real-valued features by implementing the complex operation as a two-channel real rotation, or initially use only signed cosine modulation. Compute yk = Wk inverse(A0(Wk(x))) and return y = (1/K) sum yk. Pseudocode: x -> for k: xk=W(qk,pk)x; uk=A0(xk); yk=W(qk,pk)^T uk; output mean(yk). The exact imported mathematics is unitary conjugation and expectation; q and p distributions, K, and h are engineering choices. Estimate no spectral quantities: verify numerically that each Wk preserves the feature norm and that sampled Ak has the same singular values as A0. Begin with CIFAR-10, a small ResNet-18, and a baseline with identical parameter count and FLOPs. Compare K=1 stochastic training, K=2 averaging, and no conjugation at equal training budgets. Success means improved corruption and translation robustness and validation accuracy, or reduced train-test gap, without exploding activation norms; report the extra wall-clock cost and test whether narrow noise beats ordinary additive feature noise.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.