# Quadrature-Whitened Neural Feature Subspace

- ID: 2579
- Canonical URL: https://synthcore.org/idea/2579/quadrature-whitened-neural-feature-subspace
- API JSON: https://synthcore.org/api/idea/2579.json
- API Markdown: https://synthcore.org/api/idea/2579.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.19615](https://arxiv.org/abs/2608.19615)
- Category: architecture
- Solves: stability, memory, accuracy
- ML areas: world-model, optimizer, training-dynamics
- Math tags: linear-algebra, numerical-analysis, dynamical-systems, pde, spectral-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

## Mathematical statement

Let \(\psi(x)=(\psi_1(x),\ldots,\psi_N(x))^\top\) be a fixed neural feature vector and let \(x_i\), \(i=1,\ldots,m\), be assembly quadrature points with positive weights \(w_i\). Define \(\Psi\in\mathbb{R}^{m\times N}\) by \(\Psi_{ij}=\psi_j(x_i)\) and \(W=\operatorname{diag}(w_1,\ldots,w_m)\). The discrete mass matrix is \(M=\Psi^\top W\Psi\), which may be severely ill-conditioned. Compute the weighted SVD \(A=W^{1/2}\Psi=U\Sigma V^\top\), with singular values \(\sigma_1\geq\cdots\geq\sigma_N\geq0\). Retain \(r\) directions satisfying \(\sigma_k\geq\tau\sigma_1\), and let \(V_r\) and \(\Sigma_r\) contain the retained singular vectors and singular values. Define \(q(x)=\psi(x)^\top V_r\Sigma_r^{-1}\). Its sampled matrix \(Q=\Psi V_r\Sigma_r^{-1}\) satisfies \(Q^\top WQ=I_r\), so the retained coefficient system has an identity mass matrix. Rank truncation changes the approximation space; multiplication by \(\Sigma_r^{-1}\) only changes coordinates within that retained space. For a symmetric parabolic system \(M\dot c+Kc=f\), transformed coordinates \(a=\Sigma_rV_r^\top c\) yield \(\dot a+\widetilde K a=\widetilde f\), where \(\widetilde K=Q^\top WKQ\) and \(\widetilde f=Q^\top Wf\). If \(\widetilde K\succeq0\), implicit midpoint is contractive for zero forcing.

## Key formulas

- $$u_N(x)=\sum_{j=1}^{N}c_j\psi_j(x)=\psi(x)^\top c.$$
- $$A=W^{1/2}\Psi=U\Sigma V^\top,\qquad \Psi_{ij}=\psi_j(x_i),\qquad W=\operatorname{diag}(w_1,\ldots,w_m).$$
- $$q(x)=\psi(x)^\top V_r\Sigma_r^{-1},\qquad Q=\Psi V_r\Sigma_r^{-1},\qquad Q^\top WQ=I_r.$$
- $$\left(I+\frac{\Delta t}{2}\widetilde K\right)a^{n+1}=\left(I-\frac{\Delta t}{2}\widetilde K\right)a^n+\Delta t\,\widetilde f^{n+1/2},\qquad \widetilde K=Q^\top WKQ.$$

## Implementation notes

1. Integration point: use this as a preprocessing and parameterization layer for a fixed-feature neural PDE solver, a neural operator with a continuous spatial decoder, or a latent world model whose state is represented as \(u(x)=q(x)^\top a\). Keep the original feature network \(\psi_\theta(x)\) frozen initially; train or time-step only the reduced coefficient vector \(a\). Build the basis on an assembly quadrature grid and evaluate final metrics on a separate validation grid. 2. Pseudocode: sample assembly points \(x_i\) and positive weights \(w_i\); evaluate \(\Psi_{ij}=\psi_\theta(x_i)_j\); form \(A=W^{1/2}\Psi\); compute \(A=U\Sigma V^\top\); select all indices with \(\sigma_k\geq\tau\sigma_1\); store \(B=V_r\Sigma_r^{-1}\). At each forward call, evaluate \(\psi_\theta(x)\), compute \(q(x)=\psi_\theta(x)^\top B\), and return \(u(x)=q(x)^\top a\). For a parabolic residual, assemble \(\widetilde K=Q^\top WKQ\) and use the midpoint update, solving an \(r\times r\) system instead of an \(N\times N\) system. 3. Computed from the mathematics: the weighted SVD, rank cutoff, whitening matrix, reduced mass identity, and reduced propagator. Estimated empirically: validation error, effective rank, condition numbers, and whether the numerical stiffness matrix is positive semidefinite. Use float64 for the SVD and clamp or discard singular values below the cutoff. 4. First cheap experiment: solve the one-dimensional heat equation \(u_t=u_{xx}\) on \([0,1]\) with Dirichlet boundaries using a 256-wide random Fourier-feature or tanh dictionary. Compare raw coefficients, standard Cholesky preconditioning, and the proposed weighted-SVD basis for \(\tau\in\{10^{-3},10^{-5},10^{-7}\}\), using 64 assembly points and 512 validation points. Record mass condition number, coefficient norm, loss versus wall-clock time, and relative \(L^2\) error at matched retained rank. Success means orders-of-magnitude lower mass condition number, no exploding coefficient trajectories, and equal or lower validation error at substantially smaller \(r\).

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a weighted-SVD quadrature-whitening module and reproducible toy experiment in experiment.py. The mechanism manifested strongly: the raw sampled mass condition number was 2.98e33, while every retained basis had condition number approximately 1 and identity error around machine precision; for tau=1e-3 through 1e-7, the PSD reduced midpoint propagator stayed contractive with maximum norm ratio 1.0 across dt=1e-4 to 1.0. Rank followed the singular-value cutoff (16 at tau>=1e-5, 20 at 1e-7, 132 at 1e-9), but extreme retention caused numerical instability and the reduced heat validation was worse than the raw ridge projection, so this is a mechanism/stability win rather than an established accuracy win.

### Mechanism check

- Verdict: Built a weighted-SVD quadrature-whitening module and reproducible toy experiment in experiment.py. The mechanism manifested strongly: the raw sampled mass condition number was 2.98e33, while every retained basis had condition number approximately 1 and identity error around machine precision; for tau=1e-3 through 1e-7, the PSD reduced midpoint propagator stayed contractive with maximum norm ratio 1.0 across dt=1e-4 to 1.0. Rank followed the singular-value cutoff (16 at tau>=1e-5, 20 at 1e-7, 132 at 1e-9), but extreme retention caused numerical instability and the reduced heat validation was worse than the raw ridge projection, so this is a mechanism/stability win rather than an established accuracy win.
- Confidence: 8/10
- Limitations: The dictionary is a synthetic clustered sine dictionary, not a trained neural feature network. The baseline is a ridge projection rather than a fully matched raw-coefficient time integrator, no wall-clock/FLOP benchmark or GPU run was performed, and aggressive low-singular-value retention is visibly sensitive to float64 roundoff. The heat discretization uses derivative-based weak stiffness assembled from the same quadrature grid, so separate assembly/operator quadrature effects were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/989/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/989/bench_report.json)
- [experiment.py](https://synthcore.org/code/989/experiment.py)
- [report.md](https://synthcore.org/code/989/report.md)
- [report_bench_2026-09-01T174057.md](https://synthcore.org/code/989/report_bench_2026-09-01T174057.md)
- [results.json](https://synthcore.org/code/989/results.json)
- [Download all files as ZIP](https://synthcore.org/download/989)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.