# Resolution-adaptive spectral front end

- ID: 633
- Canonical URL: https://synthcore.org/idea/633/resolution-adaptive-spectral-front-end
- API JSON: https://synthcore.org/api/idea/633.json
- API Markdown: https://synthcore.org/api/idea/633.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.09350](https://arxiv.org/abs/2607.09350)
- Category: architecture
- Solves: accuracy, stability, scalability
- ML areas: embedding, transformer, ssm, regularization
- Math tags: spectral-theory, statistics, approximation-theory, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

## Mathematical statement

The paper studies observations $Y_i=\langle X_i,\beta\rangle_{L^2}+\varepsilon_i$ and $Z_{ij}=X_i(t_{ij})+\delta_{ij}$, where $n$ is the number of trajectories, $m$ is the number of noisy evaluations per trajectory, $\beta$ is the target functional coefficient, and $\varepsilon_i,\delta_{ij}$ are observation noises. For covariance eigenvalues decaying with exponent $\alpha$ and slope smoothness exponent $s$, the independent-design prediction risk decomposes as $n^{-(2\alpha+2s)/(2\alpha+2s+1)}+(nm)^{-(2\alpha+2s)/(4\alpha+2s+1)}$. The first term is the fully observed statistical error; the second is the cost of estimating spectral information from $nm$ noisy point evaluations, amplified by the inverse covariance operator. For a common equally spaced grid, the additional terms $m^{-(2\alpha+2s)}$ and $m^{-4\alpha}$ represent fixed-grid discretization and covariance-eigenvalue identification errors. Use these terms as a mode-selection floor. Estimate $\widehat\alpha$ from the decay of empirical spectral energies and $\widehat s$ from supervised block-energy decay, then initialize the retained-mode count with $K_0=\min\{n^{1/(2\widehat\alpha+2\widehat s+1)},(nm)^{1/(4\widehat\alpha+2\widehat s+1)},m/2\}$ and screen blocks against the estimated risk floor.

## Key formulas

- $$Y_i=\left\langle X_i,\beta\right\rangle_{L^2}+\varepsilon_i,\qquad Z_{ij}=X_i(t_{ij})+\delta_{ij},\quad j=1,\dots,m,\quad i=1,\dots,n.$$
- $$R_{\mathrm{ind}}(n,m)=n^{-\frac{2\alpha+2s}{2\alpha+2s+1}}+(nm)^{-\frac{2\alpha+2s}{4\alpha+2s+1}}.$$
- $$R_{\mathrm{common}}(n,m)=n^{-\frac{2\alpha+2s}{2\alpha+2s+1}}+(nm)^{-\frac{2\alpha+2s}{4\alpha+2s+1}}+m^{-(2\alpha+2s)}+m^{-4\alpha}.$$
- $$K_0=\min\left\{n^{\frac{1}{2\widehat\alpha+2\widehat s+1}},\ (nm)^{\frac{1}{4\widehat\alpha+2\widehat s+1}},\ \frac{m}{2}\right\},\qquad \widehat R_{\mathrm{ind}}=n^{-\frac{2\widehat\alpha+2\widehat s}{2\widehat\alpha+2\widehat s+1}}+(nm)^{-\frac{2\widehat\alpha+2\widehat s}{4\widehat\alpha+2\widehat s+1}}.$$

## Implementation notes

1. Integration point: insert the module immediately after the raw sampled-curve tensor $Z\in\mathbb R^{B\times m\times d}$ and before the transformer, SSM, or neural-operator backbone. For each channel, subtract the sample mean, apply an orthonormal real FFT or discrete cosine transform, and retain only the first $K$ frequency modes or dyadic frequency blocks. Feed the retained real and imaginary coefficients, sampling locations, and a mask to the existing backbone.

2. Pseudocode:
```text
Zc = Z - mean_over_time(Z)
C = FFT_or_DCT(Zc)
e_b = mean_batch_channel(sum(abs(C[:, frequencies_in_block_b])^2))
noise_floor = median(highest_frequency_block_energies)
alpha_hat = robust_slope(log(e_b-noise_floor) versus log(block_frequency))
s_hat = cross_validated_supervised_slope_of_target_block_energy
K0 = min(n^(1/(2*alpha_hat+2*s_hat+1)),
         (n*m)^(1/(4*alpha_hat+2*s_hat+1)), m/2)
R = n^(-(2*alpha_hat+2*s_hat)/(2*alpha_hat+2*s_hat+1)) +
    (n*m)^(-(2*alpha_hat+2*s_hat)/(4*alpha_hat+2*s_hat+1))
if common_grid:
    R += m^(-(2*alpha_hat+2*s_hat)) + m^(-4*alpha_hat)
K = largest_frequency_with_block_energy_above(c*R), capped near K0
C_selected = C[:, :max(1,K)]
output = backbone(C_selected, locations, mask)
```
Use dyadic blocks $2^b\leq k<2^{b+1}$ instead of individual frequencies to make screening stable. Estimate the noise floor from repeated sensors, known sensor variance, or the median energy of the highest-frequency blocks. The mathematical quantities computed directly are the four risk terms and the cutoff scaling; $\widehat\alpha$, $\widehat s$, and threshold constant $c$ are empirical and should be clipped to a safe range. Train with random subsampling of the same curves at several values of $m$ so the network cannot exploit one fixed grid. First test on a small 1D neural operator or transformer for Burgers or advection trajectories. Compare fixed-$K$ Fourier features, raw interpolation, and this adaptive front end at equal backbone width. Measure test MSE across train and test resolutions, especially low $m$ and high noise, as well as FFT-token count and FLOPs. Success means lower MSE and less degradation as $m$ decreases, with fewer retained spectral tokens at comparable accuracy.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a dyadic FFT resolution-adaptive spectral front end with the supplied risk terms, cutoff formula, noise-floor screening, and ridge-regression comparison in experiment.py. The math check showed the common-grid risk and grid floor decrease with increasing m, as expected. In repeated noisy-curve trials, adaptive screening helped at m=16 and 32 but fixed modes were better at m=64 and 128; the no-hint estimator also overestimated alpha and retained only 2 modes, so the promised general accuracy/resolution win was not established.

### Mechanism check

- Verdict: Built a dyadic FFT resolution-adaptive spectral front end with the supplied risk terms, cutoff formula, noise-floor screening, and ridge-regression comparison in experiment.py. The math check showed the common-grid risk and grid floor decrease with increasing m, as expected. In repeated noisy-curve trials, adaptive screening helped at m=16 and 32 but fixed modes were better at m=64 and 128; the no-hint estimator also overestimated alpha and retained only 2 modes, so the promised general accuracy/resolution win was not established.
- Confidence: 8/10
- Limitations: This is a small synthetic scalar functional-regression proxy, not a neural operator or transformer. The main MSE comparison used supplied oracle alpha and s hints rather than fully estimated exponents; only five repetitions were run, no FLOP timing was measured, and independent train/test curves rather than paired physical trajectories were used across resolutions.

## Artifacts

- [experiment.py](https://synthcore.org/code/1418/experiment.py)
- [report.md](https://synthcore.org/code/1418/report.md)
- [results.json](https://synthcore.org/code/1418/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1418)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.