△ Mechanism confirmed, baseline not beaten 2026

Quadrature-Whitened Neural Feature Subspace

Implementation & benchmark of arXiv:2608.19615 — Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Stability Memory Accuracy World model Optimizer Training dynamics Linear algebra Numerical analysis Dynamical systems Pde Spectral theory

Source paper: Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization arXiv:2608.19615 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

Formulas

$$u_N(x)=\sum_{j=1}^{N}c_j\psi_j(x)=\psi(x)^\top c.$$

$$A=W^{1/2}\Psi=U\Sigma V^\top,\qquad \Psi_{ij}=\psi_j(x_i),\qquad W=\operatorname{diag}(w_1,\ldots,w_m).$$

$$q(x)=\psi(x)^\top V_r\Sigma_r^{-1},\qquad Q=\Psi V_r\Sigma_r^{-1},\qquad Q^\top WQ=I_r.$$

$$\left(I+\frac{\Delta t}{2}\widetilde K\right)a^{n+1}=\left(I-\frac{\Delta t}{2}\widetilde K\right)a^n+\Delta t\,\widetilde f^{n+1/2},\qquad \widetilde K=Q^\top WKQ.$$

Mathematical statement

Let $\psi(x)=(\psi_1(x),\ldots,\psi_N(x))^\top$ be a fixed neural feature vector and let $x_i$, $i=1,\ldots,m$, be assembly quadrature points with positive weights $w_i$. Define $\Psi\in\mathbb{R}^{m\times N}$ by $\Psi_{ij}=\psi_j(x_i)$ and $W=\operatorname{diag}(w_1,\ldots,w_m)$. The discrete mass matrix is $M=\Psi^\top W\Psi$, which may be severely ill-conditioned. Compute the weighted SVD $A=W^{1/2}\Psi=U\Sigma V^\top$, with singular values $\sigma_1\geq\cdots\geq\sigma_N\geq0$. Retain $r$ directions satisfying $\sigma_k\geq\tau\sigma_1$, and let $V_r$ and $\Sigma_r$ contain the retained singular vectors and singular values. Define $q(x)=\psi(x)^\top V_r\Sigma_r^{-1}$. Its sampled matrix $Q=\Psi V_r\Sigma_r^{-1}$ satisfies $Q^\top WQ=I_r$, so the retained coefficient system has an identity mass matrix. Rank truncation changes the approximation space; multiplication by $\Sigma_r^{-1}$ only changes coordinates within that retained space. For a symmetric parabolic system $M\dot c+Kc=f$, transformed coordinates $a=\Sigma_rV_r^\top c$ yield $\dot a+\widetilde K a=\widetilde f$, where $\widetilde K=Q^\top WKQ$ and $\widetilde f=Q^\top Wf$. If $\widetilde K\succeq0$, implicit midpoint is contractive for zero forcing.

Implementation notes

1. Integration point: use this as a preprocessing and parameterization layer for a fixed-feature neural PDE solver, a neural operator with a continuous spatial decoder, or a latent world model whose state is represented as $u(x)=q(x)^\top a$. Keep the original feature network $\psi_\theta(x)$ frozen initially; train or time-step only the reduced coefficient vector $a$. Build the basis on an assembly quadrature grid and evaluate final metrics on a separate validation grid. 2. Pseudocode: sample assembly points $x_i$ and positive weights $w_i$; evaluate $\Psi_{ij}=\psi_\theta(x_i)_j$; form $A=W^{1/2}\Psi$; compute $A=U\Sigma V^\top$; select all indices with $\sigma_k\geq\tau\sigma_1$; store $B=V_r\Sigma_r^{-1}$. At each forward call, evaluate $\psi_\theta(x)$, compute $q(x)=\psi_\theta(x)^\top B$, and return $u(x)=q(x)^\top a$. For a parabolic residual, assemble $\widetilde K=Q^\top WKQ$ and use the midpoint update, solving an $r\times r$ system instead of an $N\times N$ system. 3. Computed from the mathematics: the weighted SVD, rank cutoff, whitening matrix, reduced mass identity, and reduced propagator. Estimated empirically: validation error, effective rank, condition numbers, and whether the numerical stiffness matrix is positive semidefinite. Use float64 for the SVD and clamp or discard singular values below the cutoff. 4. First cheap experiment: solve the one-dimensional heat equation $u_t=u_{xx}$ on $[0,1]$ with Dirichlet boundaries using a 256-wide random Fourier-feature or tanh dictionary. Compare raw coefficients, standard Cholesky preconditioning, and the proposed weighted-SVD basis for $\tau\in\{10^{-3},10^{-5},10^{-7}\}$, using 64 assembly points and 512 validation points. Record mass condition number, coefficient norm, loss versus wall-clock time, and relative $L^2$ error at matched retained rank. Success means orders-of-magnitude lower mass condition number, no exploding coefficient trajectories, and equal or lower validation error at substantially smaller $r$.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a weighted-SVD quadrature-whitening module and reproducible toy experiment in experiment.py. The mechanism manifested strongly: the raw sampled mass condition number was 2.98e33, while every retained basis had condition number approximately 1 and identity error around machine precision; for tau=1e-3 through 1e-7, the PSD reduced midpoint propagator stayed contractive with maximum norm ratio 1.0 across dt=1e-4 to 1.0. Rank followed the singular-value cutoff (16 at tau>=1e-5, 20 at 1e-7, 132 at 1e-9), but extreme retention caused numerical instability and the reduced heat validation was worse than the raw ridge projection, so this is a mechanism/stability win rather than an established accuracy win.

Agent confidence: 8/10

Baseline: Raw ridge projection: mass condition number 2.98e33 before regularization; rank 256; initial validation relative L2 error 0.0289; coefficient norm 3.52.
Idea: Quadrature-whitened basis: retained ranks 16, 20, and 132 for tau=1e-3, 1e-7, and 1e-9; reduced mass condition number approximately 1.0 with identity error approximately 1e-14; midpoint max norm ratio 1.0 for moderate cutoffs. Heat relative L2 error at T=0.2 was 6.40 for rank 16 and became unstable/overflowed at rank 132.

Limitations: The dictionary is a synthetic clustered sine dictionary, not a trained neural feature network. The baseline is a ridge projection rather than a fully matched raw-coefficient time integrator, no wall-clock/FLOP benchmark or GPU run was performed, and aggressive low-singular-value retention is visibly sensitive to float64 roundoff. The heat discretization uses derivative-based weak stiffness assembled from the same quadrature grid, so separate assembly/operator quadrature effects were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.19615) Sep 2, 2026

Significantly WORSE than the tuned baseline (+1705.5%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Manufactured_heat_pde
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.0767
Idea mean: 1.3856
Effect (Δ): 1.3089 (+1705.5%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±2230.6%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 4 configs

Limitations:

Only the manufactured 1D parabolic PDE track was tested; no larger spatial PDE, multi-dimensional domain, nonlinear PDE, wall-clock/FLOP comparison, or alternative whitening cutoffs in the final full paired protocol were evaluated. The implementation uses a frozen tanh dictionary with a trainable linear readout rather than a jointly trained deep spatial feature network, and the benchmark's ridge sweep is implemented as Adam weight decay.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Built an end-to-end matched frozen-neural-dictionary benchmark on the structurally appropriate manufactured_heat_pde track, comparing raw coefficients with quadrature-whitened coefficients under the canonical train_model loop. The mass identity prediction was confirmed from the trained benchmark system (error 3.70e-14), but test MSE was substantially worse for the idea (1.3856 versus 0.07675), with paired delta +1.3089 and permutation p=0.0081. Therefore the mechanism works algebraically but does not yield the promised benchmark improvement.

Baseline: Best baseline lr=0.003, ridge=0.0; full 8-seed test MSE mean=0.07674595, std=0.00499025; per-seed=[0.08141968,0.06942547,0.07098477,0.08333057,0.08016344,0.07751243,0.07957722,0.07155404].
Idea: Best idea setting lr=0.01, tau=0.001, ridge=0.0; full 8-seed test MSE mean=1.38563919, std=1.91728933; per-seed=[0.57850820,0.35083449,0.25561139,1.90971708,6.16149235,1.61558259,0.12886217,0.08450527].
Mechanism signature: assembly_mass_identity_error=3.70488796e-14; trained_raw_prediction_std=0.28439134; trained_whitened_prediction_std=0.58684784; confirmed=true
Paired comparison: delta_mean=+1.30889324 (positive means idea worse), idea_wins=0/8, permutation_p=0.0081, verdict='idea worse (significant)'

Benchmark result