✗ Failed on benchmark 2026

Shared-response expert ranking

Implementation & benchmark of arXiv:2608.20441 — Shared Physics Responses Recover Hidden Rankings in Neural Operator Libraries

Usefulness7/10

Difficulty6/10

Novelty8/10

Architecture Speedup Accuracy Scalability Moe routing Inference speedup Graph nn Functional analysis Linear algebra Geometry Numerical analysis

Source paper: Shared Physics Responses Recover Hidden Rankings in Neural Operator Libraries arXiv:2608.20441 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build a label-free router for a finite library of neural operators by estimating one shared physical target response from an anchor prediction and using it to rank every candidate through inner products with candidate differences. The method avoids running a full residual-based diagnostic independently for every expert and can be used either to select the best expert or to form a corrected weighted combination.

Formulas

$$R_i(t)=\left\lVert t-y_i\right\rVert_M^2,\qquad G_{ij}(t)=R_j(t)-R_i(t)=\left\lVert y_j\right\rVert_M^2-\left\lVert y_i\right\rVert_M^2+2\left\langle t,y_i-y_j\right\rangle_M.$$

$$\mathcal{D}=\operatorname{span}\{y_i-y_1:2\leq i\leq N\},\qquad t\sim t'\ \Longleftrightarrow\ t-t'\in\mathcal{D}^{\perp_M},\qquad P_{\mathcal{D}}^M t\text{ is the only target information needed for ranking}.$$

$$J_a\delta_a=-F(y_a),\qquad \hat t=y_a+\delta_a,\qquad \hat s_i=2\langle\hat t,y_i\rangle_M-\|y_i\|_M^2,\qquad \hat i=\arg\max_i\hat s_i.$$

$$\hat G_{ij}=\|y_j\|_M^2-\|y_i\|_M^2+2\langle\hat t,y_i-y_j\rangle_M;\quad \text{certify }i\succ j\text{ if }\hat G_{ij}>2\,\varepsilon_{ij},\ \varepsilon_{ij}\geq\left|\langle\hat t-t,y_i-y_j\rangle_M\right|.$$

Mathematical statement

Let $\mathcal{H}$ be a real Hilbert space of discretized fields with weighted inner product $\langle u,v\rangle_M=u^\top Mv$, where $M\succ0$ is a mass or quadrature matrix, and let $y_1,\ldots,y_N\in\mathcal{H}$ be candidate neural-network outputs for the same input. For an unavailable truth $t$, candidate $i$'s squared loss is $R_i(t)=\|t-y_i\|_M^2$. The pairwise margin favoring candidate $i$ over $j$ is $G_{ij}(t)=R_j(t)-R_i(t)=\|y_j\|_M^2-\|y_i\|_M^2+2\langle t,y_i-y_j\rangle_M$. Define the candidate-difference subspace $\mathcal{D}=\operatorname{span}\{y_i-y_1:2\le i\le N\}$. Since $y_i-y_j\in\mathcal{D}$, every ranking decision depends only on $P^M_{\mathcal{D}}t$, the $M$-orthogonal projection of $t$ onto $\mathcal{D}$; adding any vector in $\mathcal{D}^{\perp_M}$ to the target leaves all rankings unchanged. In the neural implementation, estimate a proxy $\hat t$ from one anchor output $y_a$ and one shared local physical correction $\delta_a$, for example $\hat t=y_a+\delta_a$, where a discretized governing residual $F(y)=0$ gives $J_a\delta_a=-F(y_a)$ with $J_a=DF(y_a)$, solved approximately by a few preconditioned iterations. Compute $\hat G_{ij}=\|y_j\|_M^2-\|y_i\|_M^2+2\langle\hat t,y_i-y_j\rangle_M$, or equivalently $s_i=2\langle\hat t,y_i\rangle_M-\|y_i\|_M^2$ and choose $\arg\max_i s_i$. The shared-response approximation is useful when $\hat t$'s projection onto $\mathcal{D}$ is accurate even if its full-field error is large.

Implementation notes

(1) Integration point: use this as an inference-time router over a fixed library of $N$ neural operators $f_i(x)$, or over experts in a physics-constrained MoE. For one input field or parameter vector $x$, run all inexpensive candidate forward passes to obtain fields $y_i=f_i(x)$; alternatively cache these outputs if the library is already evaluated. Compute the governing-equation residual only for one anchor output $y_a$, preferably the current best average expert or the cheapest robust baseline. (2) Pseudocode: Y=[f_i(x) for i in 1..N]; choose anchor a; r=F(Y[a]); initialize delta=0; for k in 1..K: delta <- delta + preconditioner(J_a, r + J_a delta) # approximately solves J_a delta=-r; t_hat=Y[a]+delta; for i: norm_i=Y[i]^T M Y[i]; score_i=2*t_hat^T M Y[i]-norm_i; return argmax(score_i) or softmax(score_i/tau) mixture. If a Jacobian is unavailable, use finite-difference directional derivatives, an autodiff Jacobian-vector product, or a learned residual-correction network. (3) Compute exactly from the paper's mathematics the shared scores, the difference subspace, and all $M$-weighted inner products. Estimate empirically the response correction $\delta_a$, the anchor Jacobian solve tolerance, and a confidence radius $\varepsilon_{ij}$. Build $\mathcal{D}$ with QR or randomized SVD of the matrix whose columns are $y_i-y_1$; optionally project $\hat t$ and all differences into this low-rank basis to reduce router cost. Use the margin certificate only when a validation set or residual bound supplies $\varepsilon_{ij}$; otherwise treat the scores as a ranking heuristic. (4) First experiment: use a 5-10 expert Fourier Neural Operator or U-Net operator library on 2-D Burgers or reaction-diffusion snapshots, with held-out high-resolution PDE solutions available only for evaluation. Compare independent residual evaluation for every expert, the proposed one-anchor response router, random selection, and the best fixed expert. Measure pairwise ranking accuracy, top-1 expert accuracy, relative field error, residual-evaluation wall time, and total inference cost. The method succeeds if one shared solve recovers nearly the same top-1 choice as full diagnostics while reducing physics-diagnostic cost by approximately $N\times$, and if the low-rank dimension $\operatorname{rank}(\mathcal{D})$ is substantially below the field dimension. A second ablation should deliberately corrupt $\hat t$ in directions orthogonal to $\mathcal{D}$; rankings should remain unchanged up to numerical error, directly testing the claimed quotient-space invariance.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a reproducible shared-response router toy experiment with M-weighted scores, exact margin identity, difference-subspace construction, and one-anchor physical correction. The mechanism manifested: algebraic residuals were below 1.2e-14, M-orthogonal corruption changed pairwise rankings by at most 5.1e-14 even at magnitude 100, and the adversarial ranking flip occurred between 1.0 and 1.25 times the predicted threshold. The shared exact correction selected the true best expert and reduced residual evaluations 9x, although the tiny benchmark measured only about 2.6x wall-time speedup.

Agent confidence: 9/10

Baseline: All 9 residual diagnostics: approximately 0.135 ms per loop; 9 residual evaluations; true-best candidate 2.
Idea: One shared correction plus score ranking: approximately 0.052 ms per loop; 1 residual/correction; candidate 2 selected; 9x fewer diagnostic calls and approximately 2.6x measured speedup. Formula error 1.15e-14; orthogonal pairwise invariance error <=5.1e-14; predicted flip alpha 0.02308, observed bracket [1.0,1.25]x prediction.

Limitations: Only a synthetic diagonal-linear physical residual and fixed random candidate library were tested. No learned neural operators, nonlinear PDEs, approximate Jacobian/preconditioner error, noisy response estimates, GPU timing, soft mixtures, confidence certification, or independent real-data ranking accuracy were evaluated.

How to run: python3 shared_response_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.20441) Sep 2, 2026

No significant difference vs. the tuned baseline (+66.0%, p=0.252, wins 2 / 8 paired seeds).

Benchmark: Shared_response_pde
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.0011
Idea mean: 0.0018
Effect (Δ): 0.0007 (+66.0%; negative = idea better)
Wins: 2 / 8 paired seeds
p-value: 0.252 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±95.8%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 9 configs

Limitations:

The experiment used a small manufactured 1D PDE, three MLP experts, 12 epochs, and finite expert libraries rather than a large neural operator or nonlinear 2D PDE. Wall-clock speedup was not measured; only residual evaluation count was compared. The residual correction used a fixed analytic scaling rather than an iterative Jacobian/preconditioned solve, and no soft weighted-combination variant or confidence certification was tested.

How to run: python3 run_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Built a custom structurally matched manufactured PDE track and paired neural-network benchmark using identical independently trained MLP experts. The baseline's full-diagnostic selection achieved mean test MSE 0.0010874, while shared-response ranking achieved 0.0018048 (delta +0.0007174, p=0.252, 2/8 wins), so there is no significant accuracy win. The mechanism signature matched the observed best expert on only 3/8 seeds, although diagnostic calls were reduced 3x.

Baseline: mean test MSE 0.0010874417348532006; per-seed [0.0010367966, 0.0011202694, 0.0009211542, 0.0013915685, 0.0007677504, 0.0012855530, 0.0010599738, 0.0011164680]; best_cfg lr=0.01, scale=0.5; 3 residual evaluations
Idea: mean test MSE 0.0018048072524834424; per-seed [0.0010367966, 0.0011202694, 0.0009211542, 0.0048554414, 0.0020367303, 0.0011935772, 0.0009934523, 0.0022810367]; best_cfg lr=0.01, scale=0.5; 1 residual evaluation
Idea wins: 2
Mechanism signature: map[confirmed:false diagnostic_calls:map[baseline:3 idea:1] n:8 top1_matches:3]
N pairs: 8
Paired delta: 0.0007173655176302418 (idea - baseline; positive is worse)
Permutation p value: 0.252

Benchmark result