Response-Based Spectral Degeneracy Breaking

1. Integration point: use a spectral graph-convolution or spectral positional-encoding block whose symmetric operator is built from an affinity matrix \(K\) and positive node or token weights \(M\). Replace the fixed mass matrix by \(M(f)=\operatorname{diag}(\mu_1e^{\kappa f_1},\ldots,\mu_n e^{\kappa f_n})\), with \(f\) either freshly sampled Gaussian noise during training or produced by a small bounded network from node features. Solve the smallest \(r\) generalized eigenpairs \(Ku_i=\lambda_iMu_i\), normalized so \(u_i^TMu_j=\delta_{ij}\). 2. Pseudocode: sample \(f\leftarrow\operatorname{clip}(\epsilon,-c,c)\), \(\epsilon\sim\mathcal{N}(0,\Sigma_f)\); set \(M\leftarrow\operatorname{diag}(\mu\odot\exp(\kappa f))\); compute \((\lambda,U)\leftarrow\operatorname{eigh}(K,M)\); for each close eigenvalue cluster, form \(R_{ab}=u_a^T\operatorname{diag}(f)Mu_b\), and add the gap loss. For an isolated mode, estimate its response as \(\widehat{\lambda'_i}=-\kappa\lambda_i u_i^T\operatorname{diag}(f)Mu_i\). Backpropagate through the eigensolver or stop gradients through eigenvectors and use the gap loss as a regularizer. 3. The exponential mass perturbation and response formula are taken from the paper; the Gaussian covariance, clipping bound, eigengap target \(\delta\), and cluster threshold are engineering choices. Estimate only the lowest \(r\) eigenpairs with Lanczos rather than computing the full spectrum. 4. First experiment: apply this to a 2-layer spectral GCN on Cora or a 10,000-node stochastic block model, comparing fixed weights, Gaussian perturbation without regularization, and response-based gap regularization. Measure validation accuracy, repeated-perturbation prediction variance, minimum eigengap, and eigenspace rotation \(\|UU^T-U'U'^T\|_F\). Success means lower eigenspace rotation and prediction variance at matched accuracy, or higher accuracy at equal training FLOPs.