# Ambiguity-Flat Weyl Feature Layer

- ID: 2151
- Canonical URL: https://synthcore.org/idea/2151/ambiguity-flat-weyl-feature-layer
- API JSON: https://synthcore.org/api/idea/2151.json
- API Markdown: https://synthcore.org/api/idea/2151.md
- Verification status: unverified
- Source: [arXiv:2608.11850](https://arxiv.org/abs/2608.11850)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: embedding, cnn, regularization, initialization
- Math tags: linear-algebra, harmonic-analysis, spectral-theory, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace a random cyclic filter bank or patch projection with the Weyl–Heisenberg orbit of one normalized learnable prototype. Regularize the prototype so that all nonzero shift and modulation correlations have a large and nearly equal magnitude, maximizing the smallest eigenvalue of the induced feature Gram matrix and preventing poorly observed feature directions.

## Mathematical statement

For a normalized fiducial vector \(\phi\in\mathbb{C}^d\), define Weyl–Heisenberg operators \(D_{a,b}=X^aZ^b\) on cyclic coordinates, where \(a,b\in\mathbb{Z}_d\), \((X\phi)_j=\phi_{j-1\;\mathrm{mod}\;d}\), and \((Z\phi)_j=\exp(2\pi i j/d)\phi_j\). The orbit consists of projectors \(\Pi_u=D_u|\phi\rangle\langle\phi|D_u^\dagger\), and its Gram matrix is \(G_\phi^\Pi[u,v]=\operatorname{Tr}(\Pi_u\Pi_v)\). Define the ambiguity coefficient \(\chi_\phi(u)=\langle\phi|D_u|\phi\rangle\). Weyl covariance diagonalizes the Gram matrix in the displacement basis: the nonidentity eigenvalues are \(\lambda_u=d|\chi_\phi(u)|^2\), so the stability floor is \(\lambda(\phi)=\min_{u\neq(0,0)}d|\chi_\phi(u)|^2\). The neural regularizer maximizes this floor while optionally minimizing spectral spread, using \(\mathcal{L}_{\mathrm{WH}}=-\log(\epsilon+\min_{u\neq0}|\chi_\phi(u)|^2)+\beta\operatorname{Var}_{u\neq0}(d|\chi_\phi(u)|^2)\). Here \(d\) is the patch dimension, \(u\) indexes the nonidentity shift-modulation operators, \(\epsilon>0\) prevents numerical singularities, and \(\beta\geq0\) controls flatness.

## Key formulas

- $$\Pi_u=D_u\lvert\phi\rangle\!\langle\phi\rvert D_u^\dagger,\qquad G_{\phi}^{\Pi}[u,v]=\operatorname{Tr}(\Pi_u\Pi_v).$$
- $$\chi_\phi(u)=\langle\phi\rvert D_u\lvert\phi\rangle,\qquad \lambda_u=d\lvert\chi_\phi(u)\rvert^2\quad (u\neq 0).$$
- $$\lambda(\phi)=\min_{u\neq 0}d\lvert\chi_\phi(u)\rvert^2,\qquad \mathcal{L}_{\mathrm{WH}}=-\log\!\left(\epsilon+\min_{u\neq0}\lvert\chi_\phi(u)\rvert^2\right)+\beta\operatorname{Var}_{u\neq0}\!\left(d\lvert\chi_\phi(u)\rvert^2\right).$$
- $$\mathcal{F}_\phi(x)=\left(\langle D_u\phi,x\rangle\right)_{u\in\mathbb{Z}_d^2},\qquad \kappa_\phi=\frac{\max_{u\neq0}d\lvert\chi_\phi(u)\rvert^2}{\min_{u\neq0}d\lvert\chi_\phi(u)\rvert^2+\epsilon}.$$

## Implementation notes

(1) Integration point: insert this layer before a linear classifier or before the first nonlinear block in a small CNN or MLP. Split each input patch or channel vector into blocks of length d. For each block x, produce d squared complex responses y_u=<D_u phi,x>; concatenate real and imaginary parts, or use magnitudes and phases if the target architecture cannot handle complex tensors. The prototype phi is either fixed after offline optimization or learned jointly. Enforce ||phi||_2=1 after every optimizer update.

(2) Pseudocode:
```text
initialize complex phi from a random normal distribution
phi <- phi / l2_norm(phi)
for each training step:
    for u=(a,b) in Z_d^2:
        shifted <- cyclic_shift(phi, a)
        displaced <- elementwise_multiply(exp(2*pi*i*j*b/d), shifted)
        chi[u] <- conjugate(phi)^T @ displaced
    nonzero <- all u except (0,0)
    gram_eigs <- d * abs(chi[nonzero])^2
    soft_floor <- -tau * logsumexp(-gram_eigs / tau)
    L_wh <- -log(eps + soft_floor) + beta * variance(gram_eigs)
    y <- all_correlations_with_WH_orbit(x, phi)
    loss <- task_loss(network(y), target) + gamma * L_wh
    backpropagate(loss)
    optimizer_step()
    phi <- phi / l2_norm(phi)
```

(3) The quantities computed directly from the paper are cyclic shifts, phase modulations, ambiguity coefficients, and the predicted Gram eigenvalues. Compute all coefficients explicitly for d<=64; for larger d, use FFTs over the cyclic index. During warmup use the soft minimum shown above because a hard minimum gives sparse and discontinuous gradients; evaluate the hard minimum for final reporting. Also estimate the empirical feature covariance spectrum from a held-out minibatch, because the theoretical frame spectrum does not include the data distribution.

(4) First cheap experiment: use d=16 or d=32 patch vectors on CIFAR-10. Compare a learned dense projection, a random cyclic filter bank, and the Weyl–Heisenberg layer with fixed, jointly learned, and WH-regularized prototypes. Match feature count, parameter count, optimizer, batch size, and FLOPs. Measure training loss versus optimizer steps, validation accuracy, smallest singular value of the empirical feature matrix, condition number, gradient norm variance, and frozen-feature linear-probe accuracy. The main falsifiable prediction is a larger empirical singular-value floor, faster linear-probe fitting, and more stable gradient norms at equal feature dimension. If the WH regularizer only increases the theoretical floor but does not improve empirical conditioning or task accuracy, reject the method.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.