Block-Probed Rational Spectral Layer
Implementation & benchmark of arXiv:2607.10377 — Convergence analysis of a nonlinear eigensolver based on rational approximation of the resolvent
Source paper: Convergence analysis of a nonlinear eigensolver based on rational approximation of the resolvent arXiv:2607.10377 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a polynomial graph filter or repeated matrix multiplications in a graph neural network with a small rational filter evaluated at several shifts. Treat the incoming feature matrix as a block of probes rather than processing scalar probe vectors independently, allowing one set of shifted solves to expose multiple spectral directions simultaneously. The expected gain is higher approximation quality at the same number of operator applications, especially when the target filter has sharp or resonant spectral behavior.
Formulas
Mathematical statement
The paper starts from a regular nonlinear eigenproblem T(lambda)v=0 and a sketched resolvent H(z)=Omega_L^H T(z)^(-1) Omega_R, where T(z) is an n-by-n holomorphic matrix function, Omega_L and Omega_R are n-by-b probing matrices, and b is the block size. Its rational approximation represents spectral information through poles. The neural adaptation uses a fixed graph or state-space operator L in place of T and computes a rational filter q(L)X, where X in R^(n-by-d) is the feature block and d is the feature width. The filter is q(s)=sum_{j=1}^K alpha_j/(s-z_j), where z_j are distinct shifts and alpha_j are trainable coefficients. The block-probing principle is that X supplies d simultaneous probes; using d greater than one preserves multiple spectral directions, while a scalar probe can miss them. A barycentric alternative is q(s)=[sum_k w_k f_k/(s-z_k)]/[sum_k w_k/(s-z_k)], where z_k are support points, f_k are sampled values, and w_k are barycentric weights.
Implementation notes
Integrate this after graph message aggregation or inside a linear state-space block. Let L be a symmetric normalized graph Laplacian, adjacency operator, or frozen SSM transition matrix, and let X have shape [batch,n,d]. Choose K=4 to 8 real shifts z_j outside the estimated spectrum. For every training step, compute Y_j by solving (L-z_j I)Y_j=X for all d right-hand sides; use conjugate gradients for symmetric L or a reusable sparse LU factorization when L is fixed. Form Y=sum_j alpha_j Y_j, then apply a learned channel projection and residual connection: output=MLP(concat(X,Y)). Initialize alpha_j by least-squares fitting a desired low-pass, band-pass, or heat-kernel response on K+20 scalar spectral samples. Pseudocode is: Y=0; for j in range(K): Yj=solve(L-z[j]*I,X); Y += alpha[j]*Yj; out=W2*gelu(W1*concat(X,Y)). The paper's block probe is represented by X, so channels should remain a block rather than being reduced to a scalar probe. Estimate the spectral interval using 10-step Lanczos or power iteration; the mathematical quantities are the shifted resolvents and rational coefficients, while the interval estimate is empirical. First test on Cora or a synthetic stochastic-block graph with a two-layer GCN baseline, equalizing sparse-matrix-vector products. Compare K=4 rational filters with 8- or 16-step Chebyshev filters. Success means higher validation accuracy at equal operator applications, or equal accuracy with at least 2x fewer propagation steps. Also compare block processing against independently processed scalar channels.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.