✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Implementation & benchmark of arXiv:2607.09350 — The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation

Usefulness6/10

Difficulty5/10

Novelty6/10

Architecture Accuracy Stability Scalability Embedding Transformer Ssm Regularization Spectral theory Statistics Approximation theory Linear algebra

Source paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Formulas

$$Y_i=\left\langle X_i,\beta\right\rangle_{L^2}+\varepsilon_i,\qquad Z_{ij}=X_i(t_{ij})+\delta_{ij},\quad j=1,\dots,m,\quad i=1,\dots,n.$$

$$R_{\mathrm{ind}}(n,m)=n^{-\frac{2\alpha+2s}{2\alpha+2s+1}}+(nm)^{-\frac{2\alpha+2s}{4\alpha+2s+1}}.$$

$$R_{\mathrm{common}}(n,m)=n^{-\frac{2\alpha+2s}{2\alpha+2s+1}}+(nm)^{-\frac{2\alpha+2s}{4\alpha+2s+1}}+m^{-(2\alpha+2s)}+m^{-4\alpha}.$$

$$K_0=\min\left\{n^{\frac{1}{2\widehat\alpha+2\widehat s+1}},\ (nm)^{\frac{1}{4\widehat\alpha+2\widehat s+1}},\ \frac{m}{2}\right\},\qquad \widehat R_{\mathrm{ind}}=n^{-\frac{2\widehat\alpha+2\widehat s}{2\widehat\alpha+2\widehat s+1}}+(nm)^{-\frac{2\widehat\alpha+2\widehat s}{4\widehat\alpha+2\widehat s+1}}.$$

Mathematical statement

The paper studies observations $Y_i=\langle X_i,\beta\rangle_{L^2}+\varepsilon_i$ and $Z_{ij}=X_i(t_{ij})+\delta_{ij}$, where $n$ is the number of trajectories, $m$ is the number of noisy evaluations per trajectory, $\beta$ is the target functional coefficient, and $\varepsilon_i,\delta_{ij}$ are observation noises. For covariance eigenvalues decaying with exponent $\alpha$ and slope smoothness exponent $s$, the independent-design prediction risk decomposes as $n^{-(2\alpha+2s)/(2\alpha+2s+1)}+(nm)^{-(2\alpha+2s)/(4\alpha+2s+1)}$. The first term is the fully observed statistical error; the second is the cost of estimating spectral information from $nm$ noisy point evaluations, amplified by the inverse covariance operator. For a common equally spaced grid, the additional terms $m^{-(2\alpha+2s)}$ and $m^{-4\alpha}$ represent fixed-grid discretization and covariance-eigenvalue identification errors. Use these terms as a mode-selection floor. Estimate $\widehat\alpha$ from the decay of empirical spectral energies and $\widehat s$ from supervised block-energy decay, then initialize the retained-mode count with $K_0=\min\{n^{1/(2\widehat\alpha+2\widehat s+1)},(nm)^{1/(4\widehat\alpha+2\widehat s+1)},m/2\}$ and screen blocks against the estimated risk floor.

Implementation notes

1. Integration point: insert the module immediately after the raw sampled-curve tensor $Z\in\mathbb R^{B\times m\times d}$ and before the transformer, SSM, or neural-operator backbone. For each channel, subtract the sample mean, apply an orthonormal real FFT or discrete cosine transform, and retain only the first $K$ frequency modes or dyadic frequency blocks. Feed the retained real and imaginary coefficients, sampling locations, and a mask to the existing backbone.

2. Pseudocode:

Zc = Z - mean_over_time(Z)
C = FFT_or_DCT(Zc)
e_b = mean_batch_channel(sum(abs(C[:, frequencies_in_block_b])^2))
noise_floor = median(highest_frequency_block_energies)
alpha_hat = robust_slope(log(e_b-noise_floor) versus log(block_frequency))
s_hat = cross_validated_supervised_slope_of_target_block_energy
K0 = min(n^(1/(2*alpha_hat+2*s_hat+1)),
         (n*m)^(1/(4*alpha_hat+2*s_hat+1)), m/2)
R = n^(-(2*alpha_hat+2*s_hat)/(2*alpha_hat+2*s_hat+1)) +
    (n*m)^(-(2*alpha_hat+2*s_hat)/(4*alpha_hat+2*s_hat+1))
if common_grid:
    R += m^(-(2*alpha_hat+2*s_hat)) + m^(-4*alpha_hat)
K = largest_frequency_with_block_energy_above(c*R), capped near K0
C_selected = C[:, :max(1,K)]
output = backbone(C_selected, locations, mask)

Use dyadic blocks $2^b\leq k<2^{b+1}$ instead of individual frequencies to make screening stable. Estimate the noise floor from repeated sensors, known sensor variance, or the median energy of the highest-frequency blocks. The mathematical quantities computed directly are the four risk terms and the cutoff scaling; $\widehat\alpha$, $\widehat s$, and threshold constant $c$ are empirical and should be clipped to a safe range. Train with random subsampling of the same curves at several values of $m$ so the network cannot exploit one fixed grid. First test on a small 1D neural operator or transformer for Burgers or advection trajectories. Compare fixed-$K$ Fourier features, raw interpolation, and this adaptive front end at equal backbone width. Measure test MSE across train and test resolutions, especially low $m$ and high noise, as well as FFT-token count and FLOPs. Success means lower MSE and less degradation as $m$ decreases, with fewer retained spectral tokens at comparable accuracy.

Verification

Mechanism failed

Stage 1 · Toy mechanism gate: Failed ✗

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a dyadic FFT resolution-adaptive spectral front end with the supplied risk terms, cutoff formula, noise-floor screening, and ridge-regression comparison in experiment.py. The math check showed the common-grid risk and grid floor decrease with increasing m, as expected. In repeated noisy-curve trials, adaptive screening helped at m=16 and 32 but fixed modes were better at m=64 and 128; the no-hint estimator also overestimated alpha and retained only 2 modes, so the promised general accuracy/resolution win was not established.

Agent confidence: 8/10

Baseline: Fixed Fourier ridge regression MSE means at m=16/32/64/128: 0.01603/0.01157/0.00894/0.00831; retained modes: 8/12/12/12.
Idea: Adaptive Fourier ridge regression with alpha=1.2 and s=0.8 hints MSE means at m=16/32/64/128: 0.01556/0.01138/0.00912/0.00856; retained modes: 3/3/3/3. Without hints, estimated alpha was approximately 2.96 and the selector retained 2 modes.

Limitations: This is a small synthetic scalar functional-regression proxy, not a neural operator or transformer. The main MSE comparison used supplied oracle alpha and s hints rather than fully estimated exponents; only five repetitions were run, no FLOP timing was measured, and independent train/test curves rather than paired physical trajectories were used across resolutions.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 3 files · code, reports and structured results

🐍 experiment.py 6.0 KB View Raw
📄 report.md 1.6 KB View
📊 results.json 1.8 KB View Raw