Resolvent Fractional-Power Layer

Integrate this at a feature transformation or residual block, not at the scalar activation: given token or sample features X∈R^{B×d}, use a shared d×d operator A and return Y=X(A^r)^T, optionally followed by a learned projection and residual addition. Parameterize A in real arithmetic as A=S+K, with L∈R^{d×q}, S=LL^T+δI, and K=(U−U^T)/2; this guarantees the symmetric part S is positive definite while allowing nonnormal dynamics. Choose ε≥0 and m=4–12 logarithmically spaced shifts λ_j between λ_min and λ_max. Precompute quadrature nodes and weights for the scalar integral on [λ_min,λ_max], or fit them by minimizing the scalar approximation error |t^r−(sin(πr)/π)Σ_j w_j λ_j^{r−1}t/(λ_j+t)| over t∈[δ,Λ]. For each forward pass, solve (λ_jI+A_ε)Z_j=A_εX^T using a batched direct solve for d≤128 or a fixed number of preconditioned GMRES iterations for larger d, then return (sin(πr)/π)Σ_j w_jλ_j^{r−1}Z_j. Estimate δ and Λ from the eigenvalues or singular values of a detached copy of S every few hundred steps; do not differentiate through this estimation initially. The paper supplies the resolvent-mixture formula and accretive construction; quadrature truncation, node placement, and solve tolerances are empirical engineering choices. First test on CIFAR-10 with a 6-layer MLP or small Mixer at width d=64. Compare ordinary linear residual blocks, symmetric PSD fractional-power blocks using eigh, and this accretive resolvent block at matched parameter count and FLOPs. Measure training loss at equal updates, validation accuracy, gradient-norm variance, and failure rate from exploding or NaN solves. A useful positive signal is lower gradient variance or faster loss descent at equal width, with approximation error below 1e-3 on held-out scalar spectral points and no increase in forward instability. For a larger test, replace the channel-mixing matrix in a small Transformer with this layer and check perplexity versus wall-clock time and solver iterations.