# SURE-Adaptive Derivative Front End

- ID: 285
- Canonical URL: https://synthcore.org/idea/285/sure-adaptive-derivative-front-end
- API JSON: https://synthcore.org/api/idea/285.json
- API Markdown: https://synthcore.org/api/idea/285.md
- Verification status: unverified
- Source: [arXiv:2606.31776](https://arxiv.org/abs/2606.31776)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: ssm, rnn, training, neural-operator
- Math tags: numerical-analysis, statistics, linear-algebra, optimization
- Ratings: usefulness 5/10; difficulty 3/10; novelty 6/10

## Idea description

Prepend an adaptive Savitzky-Golay derivative bank to a temporal neural network. For each input channel and derivative order, select the local window by minimizing Stein's unbiased risk estimate, then concatenate the raw signal with the estimated derivatives. This supplies denoised velocity and acceleration features without requiring clean derivative targets or forcing the backbone to learn unstable finite-difference filters.

## Mathematical statement

The paper writes a differential model as $$u_t=\sum_{k=1}^{K}c_k\mathcal{F}_k\quad\text{or}\quad\mathbf{b}=\mathbf{F}\mathbf{c},$$ where $u$ is the observed field or signal, $\mathcal{F}_k$ are derivative-based features, $c_k$ are coefficients, $\mathbf{b}$ is the time-derivative data, and $\mathbf{F}$ is the feature matrix. Its derivative estimator is the local convolution $$g^{(s)}(x_n)\approx\sum_{\ell=-L}^{L}c_{s,\ell}y_{n+\ell},$$ where $y_n$ is a noisy sample, $s$ is the derivative order, $L$ is the half-window, and $c_{s,\ell}$ are polynomial differentiation weights. For polynomial degree $M$, let $V_L\in\mathbb{R}^{(2L+1)    imes(M+1)}$ have entries $(V_L)_{\ell,j}=\ell^j$ for $\ell=-L,\ldots,L$ and $j=0,\ldots,M$. Let $q_s\in\mathbb{R}^{M+1}$ satisfy $(q_s)_j=s!$ when $j=s$ and $(q_s)_j=0$ otherwise. The SG weights are $$c_{s,L}=\frac{1}{(\Delta t)^s}V_L(V_L^TV_L)^{-1}q_s,$$ where $\Delta t$ is the sample spacing. If $A_{s,L}$ is the matrix implementing this convolution and the noise is independent Gaussian with variance $\sigma^2$, SURE is $$\operatorname{SURE}_s(L)=\|A_{s,L}y-y\|_2^2+2\sigma^2\operatorname{tr}(A_{s,L})-N\sigma^2.$$ Select $$L^*=\arg\min_{L\in\mathcal{L}}\operatorname{SURE}_s(L),$$ where $\mathcal{L}$ is a finite set of candidate half-windows. The residual term measures fit to the observed signal and the trace term penalizes noise amplification; the polynomial construction preserves derivatives of polynomials up to degree $M$.

## Key formulas

- $$u_t=\sum_{k=1}^{K}c_k\mathcal{F}_k\quad\text{as}\quad\mathbf{b}=\mathbf{F}\mathbf{c}.$$
- $$g^{(s)}(x_n)\approx\sum_{\ell=-L}^{L}c_{s,\ell}\,y_{n+\ell},\qquad s=0,1,\dots,A.$$
- $$c_{s,L}=\frac{1}{(\Delta t)^s}V_L(V_L^TV_L)^{-1}q_s,\quad (V_L)_{\ell,j}=\ell^j,\quad (q_s)_j=s!\,\mathbf{1}_{j=s}.$$
- $$\operatorname{SURE}_s(L)=\|A_{s,L}y-y\|_2^2+2\sigma^2\operatorname{tr}(A_{s,L})-N\sigma^2,\qquad L^*=\arg\min_{L\in\mathcal{L}}\operatorname{SURE}_s(L).$$

## Implementation notes

Integrate the method immediately after the input projection of a GRU, temporal convolution, Mamba-like state-space model, or neural operator. For every input channel and derivative order $s\in\{1,2\}$, precompute SG convolution kernels for candidate odd widths $W=2L+1$, for example $W\in\{5,7,9,13,17\}$, using polynomial degree $M=3$ or $M=5$ and the Vandermonde formula. Apply each candidate kernel with reflection padding. Estimate the noise variance for channel $r$ using $\widehat{\sigma}_r^2=(\operatorname{median}_n|\Delta y_{r,n}-\operatorname{median}(\Delta y_r)|/0.6745)^2/2$, where $\Delta y_n=y_{n+1}-y_n$. For each candidate, compute the residual term in SURE and the trace term; for interior rows, the trace can be obtained analytically from the center convolution coefficient, while boundary rows can be included by explicitly constructing the padded linear operator. Choose the minimum-SURE candidate separately for each channel and order, using stop-gradient through the discrete argmin. Refresh choices once per epoch or smooth scores with an exponential moving average to prevent jitter. Concatenate the raw input, first derivative, and second derivative and feed them to the unchanged backbone. The mathematical components are the SG coefficients and SURE objective; variance estimation, padding, and refresh frequency are empirical choices. First evaluate on noisy Lorenz, Kuramoto-Sivashinsky, or 1D Navier-Stokes trajectories. Compare raw inputs, fixed central differences, fixed-window SG, and SURE-adaptive SG at equal model size. Track derivative MSE against clean trajectories, short- and long-horizon forecasting error, gradient-norm variance, and training failures. Success is lower forecasting error as Gaussian noise increases, at least 10 percent lower derivative MSE than central differences, and no meaningful increase in FLOPs.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.