# Summed Resolvent Spectral Layer

- ID: 1659
- Canonical URL: https://synthcore.org/idea/1659/summed-resolvent-spectral-layer
- API JSON: https://synthcore.org/api/idea/1659.json
- API Markdown: https://synthcore.org/api/idea/1659.md
- Verification status: unverified
- Source: [arXiv:2608.01155](https://arxiv.org/abs/2608.01155)
- Category: architecture
- Solves: stability, speedup, accuracy
- ML areas: graph-nn, transformer, training, inference-speedup
- Math tags: spectral-theory, linear-algebra, numerical-analysis, approximation-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 6/10

## Idea description

Replace a recursive product implementation of a rational spectral filter with an additive sum of independently evaluated resolvents. Use the layer on a graph Laplacian, token-similarity operator, or other sparse feature operator to obtain a high-order filter without multiplicative roundoff and gradient amplification; the independent solves can also be batched or distributed across devices.

## Mathematical statement

The paper defines the exact downstream spectral projector by the contour integral $\mathbf{P}^{+}=\frac{1}{2\pi}\oint_{\Gamma^{+}}(\mathbf{B}-i\alpha\mathbf{A})^{-1}\mathbf{A}\,\mathrm{d}\alpha$, where $\mathbf{A}$ and $\mathbf{B}$ form a matrix pencil, $\Gamma^{+}$ encloses the selected spectral region, and $\mathbf{P}^{+}$ is idempotent: $(\mathbf{P}^{+})^{2}=\mathbf{P}^{+}$. The paper approximates this contour projector by a rational function with poles and weights. Its central numerical result is that the same rational approximation can be evaluated either as a recursive product of $N$ resolvent factors or as an additive partial-fraction sum; the product can amplify floating-point errors multiplicatively, while the sum does not. For an ML operator $\mathbf{L}$ with eigenvalue $\lambda$, use $r_N(\lambda)=\sum_{j=1}^{N}w_j(\lambda-p_j)^{-1}$, where $p_j\in\mathbb{C}$ are poles and $w_j\in\mathbb{C}$ are partial-fraction weights. The layer applies $r_N(\mathbf{L})\mathbf{X}=\sum_{j=1}^{N}w_j(\mathbf{L}-p_j\mathbf{I})^{-1}\mathbf{X}$, with $\mathbf{X}$ the input feature matrix. Conjugate pole pairs can be combined as $2\operatorname{Re}[w_j(\mathbf{L}-p_j\mathbf{I})^{-1}\mathbf{X}]$ to keep real outputs. The projector identity motivates the optional regularizer $\|\widehat{P}^{2}\mathbf{X}-\widehat{P}\mathbf{X}\|_{F}^{2}$, where $\widehat{P}$ is the learned or fixed rational filter.

## Key formulas

- $$\mathbf{P}^{+}=\frac{1}{2\pi}\oint_{\Gamma^{+}}\left(\mathbf{B}-i\alpha\,\mathbf{A}\right)^{-1}\mathbf{A}\,\mathrm{d}\alpha,\qquad (\mathbf{P}^{+})^{2}=\mathbf{P}^{+}.$$
- $$r_N(\mathbf{L})\mathbf{X}=\sum_{j=1}^{N}w_j\left(\mathbf{L}-p_j\mathbf{I}\right)^{-1}\mathbf{X},\qquad r_N(\lambda)=\sum_{j=1}^{N}\frac{w_j}{\lambda-p_j}.$$
- $$\mathcal{L}_{\mathrm{idemp}}=\left\|\widehat{P}\,\widehat{P}\,\mathbf{X}-\widehat{P}\,\mathbf{X}\right\|_{F}^{2},\qquad \widehat{P}=r_N(\mathbf{L})\ \text{or}\ \widehat{P}=\frac{r_N(\mathbf{L})}{\max_{\lambda\in\mathcal{S}}|r_N(\lambda)|}.$$

## Implementation notes

Integrate this as a graph-nn or token-mixing layer between ordinary message-passing or attention blocks. Construct a sparse symmetric operator $\mathbf{L}$ from the batch graph Laplacian, normalized token-affinity Laplacian, or a fixed diffusion operator; add a small diagonal shift if needed so every $\mathbf{L}-p_j\mathbf{I}$ is nonsingular. Precompute $N$ poles $p_j$ and weights $w_j$ from a target spectral split, for example by fitting $r(\lambda)\approx 1$ on low eigenvalues and $r(\lambda)\approx 0$ on high eigenvalues over an estimated interval $[\lambda_{\min},\lambda_{\max}]$. Pseudocode: input $X$; for each pole $j$ in parallel, solve $(L-p_jI)Y_j=X$ using sparse factorization or fixed-iteration preconditioned CG/GMRES; accumulate $Z=\sum_jw_jY_j$; take the real part; apply a learned channel matrix $ZV$ and residual connection $X+\gamma ZV$. For complex poles, process conjugate pairs together. The mathematical computation is the additive resolvent evaluation; estimate $\lambda_{\max}$ by 10-step power iteration and measure solve residuals $\|(L-p_jI)Y_j-X\|/\|X\|$ empirically. First experiment: train a 4-layer GCN on Cora or a small 2D mesh node-classification problem with $N\in\{4,8,16,32\}$, identical poles, weights, optimizer, and FLOP budget. Compare the summed implementation with a recursive product implementation of the same rational filter and with a Chebyshev polynomial baseline. Track validation accuracy, gradient norm variance, filter error against a high-precision reference, solver residuals, and wall-clock time. Success means the summed version remains accurate as $N$ grows, avoids NaNs or exploding gradients seen in the product version, and improves accuracy or loss at equal solve cost; a secondary test should measure scaling when the independent solves are placed in separate GPU streams or batched sparse-solve kernels.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.