✗ Failed on benchmark 2026

Spectral-gap adaptive polynomial filtering

Implementation & benchmark of arXiv:2607.24643 — Anderson acceleration of the proximal point method: the exact adaptive minimax, a spectral phase transition, and optimal safeguarding

Usefulness7/10

Difficulty7/10

Novelty7/10

Dynamics Speedup Stability Accuracy Optimizer Training dynamics Ssm Diffusion sampling Spectral theory Polynomial approximation Dynamical systems Numerical analysis

Source paper: Anderson acceleration of the proximal point method: the exact adaptive minimax, a spectral phase transition, and optimal safeguarding arXiv:2607.24643 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the paper's sharp sK approximately equal to 1 phase transition to choose between conservative Fejer averaging and higher-order polynomial filtering. When the local fixed-point spectrum is separated from eigenvalue 1, use a Jackson-type filter; near the critical regime, use the safe Fejer filter instead of unrestricted Anderson extrapolation.

Formulas

$$s=\operatorname{dist}(1,\sigma(L)).$$

$$\|r(y_K)\|=O\!\left(\frac{d_0}{K^2s}\right)\quad\text{when }sK\to\infty.$$

$$s_{\mathrm{crit}}=\sin\!\left(\frac{\pi}{2K+2}\right)\asymp\frac{1}{K},\qquad \|r(y_K)\|_{\mathrm{minimax}}=\frac{d_0}{K+1}.$$

$$\widehat y_K^{\mathrm{safe}}=\frac{1}{K+1}\sum_{j=0}^{K}(2J-I)^j y_0.$$

Mathematical statement

Let L be the local linearized resolvent or fixed-point operator, sigma(L) its spectrum, and s=dist(1,sigma(L)) the spectral distance from the slow mode at 1. The paper states that a Jackson-kernel polynomial achieves residual O(d_0/(K^2 s)) when sK tends to infinity. At the critical scale s_crit=sin(pi/(2K+2)), which is asymptotically proportional to 1/K, the extremal instance still has exact minimax residual d_0/(K+1). Thus high-order acceleration is justified only when the estimated gap satisfies sK substantially greater than one. The safe baseline is the Fejer estimator y_hat_K=(K+1)^(-1) sum from j=0 to K of (2J-I)^j y_0.

Implementation notes

Apply this to the solver inside a DEQ, recurrent state-space block, proximal transformer layer, or iterative diffusion sampler. At state y, define F(y)=J(y) and estimate the local Jacobian L=DF(y) with matrix-free automatic differentiation. Draw m=8 random Rademacher vectors v_i and compute L v_i using Jacobian-vector products. For a cheap gap proxy, estimate the largest eigenvalue of L near 1 by three to five power iterations, or form the small Krylov matrix from vectors v, Lv, L^2v and estimate the smallest singular value of I-L on that subspace. Set the conservative gap estimate s_hat to the clipped value in [0,1]. Select K from {3,7,15}. If s_hat*K<2, use the safe Fejer block: z_0=y, z_(j+1)=2J(z_j)-z_j, and y_out=(K+1)^(-1) sum_j z_j. If s_hat*K>=2, apply a precomputed Jackson polynomial p(t)=sum from j=0 to K of c_j t^j with p(1)=1 to the same iterates; validate the coefficients first on diagonal matrices with known spectra. Reject any filtered output whose normalized residual increases by more than 10 percent, and fall back to Fejer averaging. The first cheap experiment should use a 1M-parameter DEQ MLP on CIFAR-10, with plain iteration and always-on Anderson acceleration as baselines. Measure residual reduction per J evaluation, solver failures, wall-clock cost, and task accuracy separately for gapped and near-critical synthetic Jacobian spectra. The target is at least 1.5x fewer evaluations in the gapped regime with no worse stability near s_hat*K approximately 1.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable toy MVP for Fejer and Jackson-type polynomial filtering on exact rotation spectra, including coefficient validation, explicit iterate equivalence, adaptive selection, and a 10% residual safeguard. The mechanism manifested: the critical residual matched 1/(K+1) to approximately 1e-6 or better, while fixed sK=8 produced nearly constant K^2*s*residual around 3.4, consistent with O(1/(K^2 s)). The nominal sK=2 switch was too aggressive on this spectrum, but the safeguard rejected the approximately 1.45x degradation; Jackson became beneficial around sK=3.5 and was substantially better by sK=8.

Agent confidence: 8/10

Baseline: Fejer critical residual*(K+1) = 0.9999998 mean, 0.9999999993 maximum; plain degree-K iteration residual ratio = 1.0 on the rotation-envelope measurement.
Idea: Jackson fixed-sK=8 residual*K^2*s = 3.36, 3.41, 3.37, 3.39, 3.41 for K=7,15,31,63,127; at sK=8 it reduced residual by roughly 2.3x-2.5x versus Fejer. At sK=2 its residual was about 1.45x Fejer and was rejected by the 10% safeguard.

Limitations: Only synthetic diagonal rotation spectra were tested; no matrix-free Jacobian estimation, DEQ/recurrent model, CIFAR-10 task accuracy, wall-clock evaluation budget, Anderson baseline, noisy gap estimation, or nonlinear operator behavior was tested. The Jackson construction is a simple squared-Fejer polynomial approximation rather than a full paper-derived coefficient implementation.

How to run: python3 spectral_gap_filter.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.24643) Sep 2, 2026

No significant difference vs. the tuned baseline (+35.5%, p=0.0568, wins 1 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0014
Idea mean: 0.0019
Effect (Δ): 0.0005 (+35.5%; negative = idea better)
Wins: 1 / 8 paired seeds
p-value: 0.0568 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±38.8%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the structurally matched registered dynamics track was tested. The implementation uses a fixed scalar gap proxy and a small squared-Fejer Jackson-type polynomial rather than matrix-free Jacobian estimation in a DEQ; solver evaluation counts and wall-clock speedups were not measured.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

The registered dynamics/rnn_small benchmark was rerun with 8 paired seeds and a baseline sweep over the same learning-rate union used by the idea. The adaptive filter did not win: baseline mean test MSE was 0.00141983 versus 0.00192421 for the idea, paired delta was +0.00050438, and permutation p=0.05675. The trained-model mechanism signature was not confirmed, so this is an honest negative transfer result.

Baseline: best lr=0.01; mean test MSE 0.0014198302378645167; std 0.00039839114901047844
Idea: lr=0.01, K=7, gap_hat=0.25, sK=1.75; mean test MSE 0.0019242097914684564; std 0.0006243480083057443

Benchmark result