# Basis-Invariant Spectral Block Shrinkage

- ID: 2197
- Canonical URL: https://synthcore.org/idea/2197/basis-invariant-spectral-block-shrinkage
- API JSON: https://synthcore.org/api/idea/2197.json
- API Markdown: https://synthcore.org/api/idea/2197.md
- Verification status: unverified
- Source: [arXiv:2608.12637](https://arxiv.org/abs/2608.12637)
- Category: regularization
- Solves: accuracy, memory, generalization
- ML areas: graph-nn, cnn, regularization, pruning
- Math tags: spectral-theory, linear-algebra, geometry, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Insert a proximal layer after a graph, mesh, or spherical convolution that groups all coordinates belonging to the same Laplacian eigenspace and applies one shared shrinkage gate to the whole group. Unlike coefficientwise spectral pruning, the result is unchanged if the eigenvectors inside a repeated eigenspace are rotated, preventing arbitrary basis-dependent feature selection.

## Mathematical statement

Let L\in\mathbb{R}^{n    imes n} be a symmetric graph or manifold Laplacian, and let E_b be its b-th complete eigenspace, with orthogonal projector P_b onto E_b. If U_b is any orthonormal basis of E_b, then P_b=U_bU_b^{    op}; replacing U_b by U_bQ for any orthogonal matrix Q leaves P_b unchanged. For node features H\in\mathbb{R}^{n    imes d}, the spectral block is H_b=P_bH and its magnitude is r_b=\|H_b\|_F. The block-variation penalty is V(H)=\sum_b w_b\|P_bH\|_F, where w_b\geq0 is an optional frequency weight. The proximal map of this group penalty is block soft-thresholding, G_b(H)=\left(1-\frac{    au w_b}{\|P_bH\|_F+\varepsilon}\right)_+P_bH, with threshold     au\geq0, numerical constant \varepsilon>0, and (x)_+=\max(x,0). The adapted layer is H_{\mathrm{out}}=\sum_bG_b(H), possibly retaining an unprojected residual H_{\mathrm{res}}. The key property is basis invariance: P_b and therefore \|P_bH\|_F and G_b(H) do not depend on the choice of eigenvectors within E_b.

## Key formulas

- $$P_b=U_bU_b^{\top},\qquad U_b^{\top}U_b=I,\qquad U_b\mapsto U_bQ\ \Rightarrow\ (U_bQ)(U_bQ)^{\top}=P_b.$$
- $$V(H)=\sum_{b=1}^{B}w_b\left\|P_bH\right\|_F.$$
- $$G_b(H)=\left(1-\frac{\tau w_b}{\left\|P_bH\right\|_F+\varepsilon}\right)_+P_bH,\qquad H_{\mathrm{out}}=\sum_{b=1}^{B}G_b(H).$$
- $$\operatorname*{arg\,min}_{Z}\frac{1}{2}\|Z-H\|_F^2+\tau\sum_{b=1}^{B}w_b\|P_bZ\|_F\quad\Longrightarrow\quad Z=H_{\mathrm{out}}.$$

## Implementation notes

Integrate this as a spectral pruning or regularization layer immediately after a graph convolution, mesh convolution, spherical convolution, or any tensor represented on n geometric sites. The input is H with shape [batch,n,channels]. First compute a symmetric Laplacian L for the fixed graph or mesh. For a small prototype, run an eigendecomposition L=U\Lambda U^    op, group eigenvectors whose eigenvalues differ by at most a tolerance \delta, and store one projector P_b=U_bU_b^    op per group. For each batch element, compute H_b=P_bH, r_b=sqrt(sum(H_b^2)), gate_b=relu(1-tau*w_b/(r_b+eps)), and accumulate H_out += gate_b*H_b. Use tau=softplus(tau_raw) and either fix w_b=1 or set w_b=(1+lambda_b)^s for a tunable frequency exponent s. During training, tau_raw can be learned or updated by a proximal schedule; for a cheaper differentiable version, use the gate directly and add V(H) to the task loss. The mathematical quantities are the eigenspace projectors and the exact block proximal formula. Empirically estimate the useful threshold by monitoring r_b distributions, validation accuracy, and the fraction of gates below 0.05; do not independently threshold coordinates within a block. For large graphs, replace dense projectors with truncated eigenvectors or approximate polynomial projectors, but validate invariance first on a small exact decomposition. The first experiment should use Spherical MNIST or a mesh classification benchmark with a spherical or graph CNN baseline. Compare the baseline, coefficientwise spectral pruning, and block shrinkage at equal parameter count and FLOPs. Success means equal-or-better accuracy with fewer active spectral blocks, improved robustness after randomly rotating eigenvectors inside repeated eigenspaces, and measurable activation or downstream-layer memory reduction.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.