Unverified 2026

Lp Spectral-Gap Invariance Regularizer

Implementation & benchmark of arXiv:2609.01447 — Kazhdan's Property $(T)$ for Subspaces and Quotients of $L_p$-Spaces

Usefulness5/10

Difficulty5/10

Novelty5/10

Regularization Generalization Stability Accuracy Regularization Cnn Representation theory Functional analysis Geometry Spectral theory

Source paper: Kazhdan's Property $(T)$ for Subspaces and Quotients of $L_p$-Spaces arXiv:2609.01447 ⓘ · analyzed Sep 2, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Apply a regularizer that penalizes feature disagreement under a finite set of known transformations. The paper's spectral-gap inequality gives a quantitative reason that this local consistency penalty controls distance from the subspace invariant under the transformation group, while the task loss prevents undesirable collapse.

Formulas

$$\max_{q\in Q}\|\rho(q)x-x\|_p\geq c_p\,\operatorname{dist}\bigl(x,X^{\rho(G)}\bigr).$$

$$X^{\rho(G)}=\{x\in X:\rho(g)x=x\ \text{for every }g\in G\},\qquad \operatorname{dist}(x,X^{\rho(G)})=\inf_{y\in X^{\rho(G)}}\|x-y\|_p.$$

$$\Phi(x)(\omega)=\operatorname{sgn}(x(\omega))|x(\omega)|^m,\qquad p=2m,\qquad \|\Phi(x)\|_2^2=\|x\|_{2m}^{2m}.$$

$$\mathcal L_{\mathrm{gap}}(z)=\frac{1}{|Q|}\sum_{q\in Q}\|\rho(q)z-z\|_p^p,\qquad \mathcal L_{\mathrm{inv}}(z)=\left\|z-\frac{1}{|Q|}\sum_{q\in Q}\rho(q)z\right\|_p^p.$$

Mathematical statement

The main estimate states that for a sigma-finite measure space (Omega, Sigma, mu), a closed subspace X of L_p(mu), a strongly continuous isometric representation rho from G to O(X), a finite set Q in G, and x in X, there exists c_p greater than zero such that max over q in Q of ||rho(q)x - x||_p is at least c_p times dist(x, X^{rho(G)}). Here O(X) is the group of linear L_p-isometries, X^{rho(G)} = {x in X : rho(g)x = x for every g in G} is the invariant subspace, and dist(x, X^{rho(G)}) = inf over y in X^{rho(G)} of ||x-y||_p. For even p = 2m, the paper uses the power map Phi(x)(omega) = sgn(x(omega)) |x(omega)|^m, which satisfies ||Phi(x)||_2^2 = ||x||_{2m}^{2m}, thereby converting L_{2m} magnitudes into Hilbert-space magnitudes. In a neural network, use a finite-group average as an approximate invariant projector and penalize the residual in L_p.

Implementation notes

1. Integration point: insert the regularizer after an intermediate feature tensor z = f_theta(x), before the classifier or decoder head. Choose Q as a small set of transformations such as rotations, reflections, graph automorphisms, token permutations, or domain-specific symmetries. Implement rho(q) as a permutation or signed permutation of feature coordinates whenever possible. If the transformation acts on inputs instead, compute z_q = f_theta(qx) and use z_q as the transformed representation.

2. Pseudocode: z = encoder(x); zq = [rho(q, z) for q in Q]; gap = mean([mean(abs(v - z)p) for v in zq]); zmean = mean(zq); inv = mean(abs(z - zmean)p); loss = task_loss(head(z), y) + lambda_gap * gap + lambda_inv * inv. For p = 2m, optionally compute phi(v) = sign(v) * abs(v)m and add gap2 = mean([mean((phi(v) - phi(z))2) for v in zq]), normalized by mean(phi(z)**2) + epsilon.

3. The mathematical result supplies the spectral-gap principle and the dependence on p and the representation's Kazhdan constant, but c_p will normally be unknown in an ML application. Estimate an empirical diagnostic with ratio = max_q norm_p(rho(q,z) - z) / (norm_p(z - mean_q rho(q,z)) + epsilon). The finite average is only an approximate projector unless Q covers the full group, so keep the supervised task loss and monitor covariance rank to detect feature collapse. Tune lambda_gap so that the regularizer is initially between 1 and 10 percent of the task loss.

4. First cheap experiment: use a ResNet-18 on CIFAR-10. Let Q be the four rotations, or the dihedral set of rotations and horizontal reflections, and compare a baseline, ordinary pairwise consistency loss, and the L4 spectral-gap regularizer at matched training FLOPs. Measure clean accuracy, accuracy on rotated and corrupted test images, transformation disagreement, feature covariance effective rank, and the empirical ratio. The method is supported if it improves transformed or corrupted accuracy and reduces disagreement without materially reducing clean accuracy or feature rank.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.