# Resolvent Fractional-Power Layer

- ID: 619
- Canonical URL: https://synthcore.org/idea/619/resolvent-fractional-power-layer
- API JSON: https://synthcore.org/api/idea/619.json
- API Markdown: https://synthcore.org/api/idea/619.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.09152](https://arxiv.org/abs/2607.09152)
- Category: architecture
- Solves: stability, accuracy, scalability
- ML areas: architecture, training-dynamics, optimizer
- Math tags: functional-analysis, linear-algebra, numerical-analysis, convex-analysis
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Parameterize a learned feature-space operator as accretive but not necessarily symmetric, then apply its fractional power through a finite positive mixture of shifted resolvents. This provides a matrix-function layer that can represent directional and rotational interactions while avoiding unstable eigendecomposition of nonnormal matrices.

## Mathematical statement

The paper represents an operator-monotone function f on (0,∞), with finite f(0), as f(A)=f(0)I+bA+∫_0^∞(1/λ I−(λI+A)^{-1})dμ(λ), where A is a bounded accretive operator, b≥0, and μ is a positive measure satisfying the integrability condition from the paper. For 0<r<1, the fractional power is A^r=(sin(πr)/π)∫_0^∞(1/λ I−(λI+A)^{-1})λ^r dλ. Accretivity means Re(A)=(A+A*)/2 is positive semidefinite; with εI added it is strictly accretive, so λI+A+εI is invertible for every λ>0. Using 1/λ I−(λI+A)^{-1}=λ^{-1}(λI+A)^{-1}A, the fractional power can be evaluated using shifted linear solves rather than eigenvectors. The transferable property is that the function is a positive resolvent mixture, which remains well behaved for nonnormal operators and can be approximated by a small rational sum.

## Key formulas

- $$f(A)=f(0)I+bA+\int_{0}^{\infty}\left(\frac{1}{\lambda}I-(\lambda I+A)^{-1}\right)d\mu(\lambda).$$
- $$A^{r}=\frac{\sin(\pi r)}{\pi}\int_{0}^{\infty}\left(\frac{1}{\lambda}I-(\lambda I+A)^{-1}\right)\lambda^{r}\,d\lambda,\qquad 0<r<1.$$
- $$A^{r}\approx \frac{\sin(\pi r)}{\pi}\sum_{j=1}^{m}w_j\lambda_j^{r-1}(\lambda_j I+A_\varepsilon)^{-1}A_\varepsilon,\qquad A_\varepsilon=A+\varepsilon I.$$
- $$A=S+K,\qquad S=\frac{A+A^{*}}{2}=LL^{*}+\delta I\succ0,\qquad K=\frac{A-A^{*}}{2},\quad K^{*}=-K.$$

## Implementation notes

Integrate this at a feature transformation or residual block, not at the scalar activation: given token or sample features X∈R^{B×d}, use a shared d×d operator A and return Y=X(A^r)^T, optionally followed by a learned projection and residual addition. Parameterize A in real arithmetic as A=S+K, with L∈R^{d×q}, S=LL^T+δI, and K=(U−U^T)/2; this guarantees the symmetric part S is positive definite while allowing nonnormal dynamics. Choose ε≥0 and m=4–12 logarithmically spaced shifts λ_j between λ_min and λ_max. Precompute quadrature nodes and weights for the scalar integral on [λ_min,λ_max], or fit them by minimizing the scalar approximation error |t^r−(sin(πr)/π)Σ_j w_j λ_j^{r−1}t/(λ_j+t)| over t∈[δ,Λ]. For each forward pass, solve (λ_jI+A_ε)Z_j=A_εX^T using a batched direct solve for d≤128 or a fixed number of preconditioned GMRES iterations for larger d, then return (sin(πr)/π)Σ_j w_jλ_j^{r−1}Z_j. Estimate δ and Λ from the eigenvalues or singular values of a detached copy of S every few hundred steps; do not differentiate through this estimation initially. The paper supplies the resolvent-mixture formula and accretive construction; quadrature truncation, node placement, and solve tolerances are empirical engineering choices. First test on CIFAR-10 with a 6-layer MLP or small Mixer at width d=64. Compare ordinary linear residual blocks, symmetric PSD fractional-power blocks using eigh, and this accretive resolvent block at matched parameter count and FLOPs. Measure training loss at equal updates, validation accuracy, gradient-norm variance, and failure rate from exploding or NaN solves. A useful positive signal is lower gradient variance or faster loss descent at equal width, with approximation error below 1e-3 on held-out scalar spectral points and no increase in forward instability. For a larger test, replace the channel-mixing matrix in a small Transformer with this layer and check perplexity versus wall-clock time and solver iterations.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a nonnormal accretive resolvent fractional-power layer using positive log-grid quadrature and shifted linear solves, with a principal-matrix-power and stability verification. The resolvent approximation stayed finite as skew increased, but the truncated quadrature produced 19.3% scalar error and 4.6–6.0% matrix error. On one tiny classification task it achieved 1.000 versus 0.989 validation accuracy and lower loss at step 60, but was 2.8x slower, had higher gradient-norm variance, and was not parameter- or FLOP-matched, so no demonstrated win was established.

### Mechanism check

- Verdict: Built a nonnormal accretive resolvent fractional-power layer using positive log-grid quadrature and shifted linear solves, with a principal-matrix-power and stability verification. The resolvent approximation stayed finite as skew increased, but the truncated quadrature produced 19.3% scalar error and 4.6–6.0% matrix error. On one tiny classification task it achieved 1.000 versus 0.989 validation accuracy and lower loss at step 60, but was 2.8x slower, had higher gradient-norm variance, and was not parameter- or FLOP-matched, so no demonstrated win was established.
- Confidence: 7/10
- Limitations: Only one small synthetic dataset and one seed were tested; the comparison was not parameter- or FLOP-matched, no CIFAR-10 or Transformer experiment was run, quadrature nodes were not optimized, and no large-width scalability or GMRES study was performed.

## Artifacts

- [report.md](https://synthcore.org/code/1414/report.md)
- [results.json](https://synthcore.org/code/1414/results.json)
- [run_experiment.py](https://synthcore.org/code/1414/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/1414)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.