Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Usefulness7/10
Difficulty6/10
Novelty6/10

Source paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143 · analyzed Sep 2, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Formulas

$$\frac{\partial u}{\partial t}+\frac{\partial}{\partial r}(F(u,r))=S(u,r).$$
$$s_j=\dot{x}_j=\frac{F(u_j^+,x_j)-F(u_j^-,x_j)}{u_j^+-u_j^-},\qquad x_j^{n+1}=x_j^n+\Delta t\,s_j^n.$$
$$\widehat{u}^{\,n+1}(r)=u_{\mathrm{front}}\!\left(r;\{x_j^{n+1},u_j^{-,n+1},u_j^{+,n+1}\}_{j=1}^K\right)+R_\theta\!\left(u^n,r,\{x_j^n,u_j^-,u_j^+\}_{j=1}^K\right).$$
$$\mathcal{L}=\|\widehat{u}^{\,n+1}-u_{\mathrm{target}}^{n+1}\|_1+\lambda_{\mathrm{RH}}\sum_{j=1}^K\left|\frac{x_j^{n+1}-x_j^n}{\Delta t}-\frac{F(u_j^+,x_j^n)-F(u_j^-,x_j^n)}{u_j^+-u_j^-}\right|.$$

Mathematical statement

The paper formulates a one-dimensional balance law as $\partial_t u+\partial_r F(u,r)=S(u,r)$, where $u(t,r)$ is the state, $F(u,r)$ is a possibly geometry-dependent flux, $S(u,r)$ is a source term, $t$ is time, and $r$ is space. For a discontinuity at position $x_j(t)$ with left and right traces $u_j^-$ and $u_j^+$, the adapted local Rankine–Hugoniot law is $\dot{x}_j=s_j=[F(u_j^+,x_j)-F(u_j^-,x_j)]/(u_j^+-u_j^-)$ when the source is bounded and does not contain a delta mass at the front. The source changes the states on either side but not the instantaneous jump speed; it can therefore be handled by the learned or numerical smooth branch. The proposed module exploits the exact kinematic constraint $x_j^{n+1}=x_j^n+\Delta t\,s_j^n$ while predicting only a residual field. For multiple fronts, the token set is $\{(x_j,u_j^-,u_j^+)\}_{j=1}^K$, and attention supplies each front with local context used to estimate its traces and resolve interactions.

Implementation notes

(1) Integration point: insert the module between the encoder and decoder of a 1D Fourier neural operator or token transformer for time stepping. Given a grid state $u^n\in\mathbb{R}^N$, detect candidate fronts using $|u_{i+1}^n-u_i^n|$ above a threshold, cluster adjacent flagged cells, and create one token per cluster containing position $x_j^n$, estimated traces $u_j^-$ and $u_j^+$, and local pooled features. Use cross-attention from grid points to front tokens. (2) Pseudocode: compute front tokens; for each token evaluate $s_j=(F(u_j^+,x_j)-F(u_j^-,x_j))/(u_j^+-u_j^-)$; set $x_j\leftarrow x_j+\Delta t s_j$; use a small MLP to update $u_j^-$ and $u_j^+$ from attended context; render a piecewise front field; predict a smooth residual $R_\theta$ with the FNO; return front field plus residual. Merge or delete tokens when fronts collide or their jump falls below a threshold. (3) Computed from the mathematics: the balance-law interface and the front speed. Estimated empirically: front locations, traces, detection threshold, and the residual network. If $F$ is known, evaluate it exactly; otherwise use a differentiable supplied flux model. (4) First experiment: train on scalar Burgers data $F(u)=u^2/2$, optionally with a spatial source $S(r,u)$, using 64–256 grid points and randomly varying shock/rarefaction initial conditions. Compare against an equal-parameter FNO, a transformer without front tokens, and a standard residual physics-informed model. Equalize training steps, batch size, and rollout FLOPs. Measure $L^1/L^2$ field error, shock-position error, and error after 10–50 autoregressive steps. (5) Pre-register mechanism tests: at fixed grid resolution, the proposed model must reduce median shock-position error by at least 30% relative to the FNO while having no more than 10% worse smooth-region error; its one-step front-speed error must scale approximately linearly with $\Delta t$ over three step sizes, with log-log slope in $[0.8,1.2]$; and removing only the analytical update, replacing $x_j^{n+1}$ by a learned position head, must increase long-rollout front error by at least 25%. Falsification occurs if the analytical update gives no improvement over learned positions, violates the measured Rankine–Hugoniot speed by more than 10% on clean shocks, or improves field loss only through substantially higher compute.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Stage 1 — Mechanism check agent confidence 7/10

Built a self-contained Burgers front-token MVP with jump detection, local Rankine–Hugoniot speed evaluation, analytical front propagation, piecewise rendering, and a Godunov finite-volume baseline. The RH identity was reproduced to machine precision, and analytical tokens had essentially zero front and field error on the clean shock while the baseline errors decreased with resolution. This is a clear toy signal, but it is not evidence yet for learned residuals, attention, source terms, front interactions, or a fair equal-compute neural comparison.

Agent confidence
7/10
Baseline
Mean over N={64,128,256}: front-position L1=0.00867 and terminal field L1=0.00557; per-resolution front errors=[0.01468,0.00760,0.00373].
Idea
Mean over N={64,128,256}: front-position L1=5.22e-16 and terminal field L1=0.0; RH identity maximum error=0.0; one-step position errors were 0.0 for dt={0.002,0.004,0.008}.

Limitations: The test uses a single clean scalar Burgers shock with exact constant traces, so the token branch is effectively oracle-like and does not train a neural residual or attention module. It does not test source terms, rarefactions, multiple interacting fronts, noisy detection, autoregressive learned rollouts, equal-parameter/FLOP neural baselines, or the learned-versus-analytic position ablation.

How to run: python3 front_tokens_experiment.py

Stage 2 — Benchmark vs. tuned baseline

Significantly WORSE than the tuned baseline (+1065.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Burgers_periodic_split
Model
cnn_shared
Paired seeds
8
Baseline mean
0.0002
Idea mean
0.0022
Effect (Δ)
0.002 (+1065.9%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±393.4%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

Only the registered burgers_periodic_split one-step PDE track was tested. No source terms, rarefactions, multiple interacting fronts, long autoregressive rollouts, learned-position ablation, FNO comparison, or explicit FLOP measurement was included.

How to run: python3 bench_front_tokens.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 9/10

The registered Burgers PDE bench completed with 8 paired seeds and a three-configuration baseline sweep. The front-token model had test MSE 0.0022266772 versus 0.0001909858 for the baseline; paired delta was +0.0020356914 with permutation p=0.0081, so the idea was significantly worse. The trained-model mechanism signature was not confirmed.

Baseline
MSE mean=0.0001909858247, std=0.0000489987634; best config lr=0.006, epochs=15.
Idea
MSE mean=0.002226677214, std=0.000878247893; selected config lr=0.006, epochs=15.
Paired comparison
delta_mean=+0.002035691389, idea_wins=0/8, permutation p_value=0.0081, verdict=idea worse (significant).

Benchmark result

Significantly WORSE than the tuned baseline (+1065.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Burgers_periodic_split
Model
cnn_shared
Paired seeds
8
Baseline mean
0.0002
Idea mean
0.0022
Effect (Δ)
0.002 (+1065.9%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±393.4%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

Only the registered burgers_periodic_split one-step PDE track was tested. No source terms, rarefactions, multiple interacting fronts, long autoregressive rollouts, learned-position ablation, FNO comparison, or explicit FLOP measurement was included.

How to run: python3 bench_front_tokens.py

Artifacts

⬇ Download all as ZIP 6 files · code, reports and structured results