# Parseval Scattering Stem with Certified Depth

- ID: 2494
- Canonical URL: https://synthcore.org/idea/2494/parseval-scattering-stem-with-certified-depth
- API JSON: https://synthcore.org/api/idea/2494.json
- API Markdown: https://synthcore.org/api/idea/2494.md
- Verification status: unverified
- Source: [arXiv:2608.18064](https://arxiv.org/abs/2608.18064)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: cnn, embedding, regularization
- Math tags: harmonic-analysis, functional-analysis, approximation-theory, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Replace the first several convolutional blocks of a small image model with a finite-depth convolution-modulus scattering stem built from a Parseval filter bank. Enforce exact energy accounting and use the paper's polynomial residual law to choose the smallest depth that captures the desired fraction of input energy, avoiding unstable or redundant deep scattering paths.

## Mathematical statement

The filter bank consists of a low-pass filter phi and band-pass filters psi_lambda whose Fourier multipliers satisfy the Parseval partition |widehat{phi}(xi)|^2 + sum_lambda |widehat{psi_lambda}(xi)|^2 = 1 for every frequency xi in R^d. For a path p=(lambda_1,...,lambda_m), define U[emptyset]f=f and U[(p,lambda)]f = |U[p]f * psi_lambda|, where * is convolution and |.| is pointwise modulus; the scattering coefficient is S[p]f = U[p]f * phi. The modulus is nonexpansive, || |u|-|v| ||_2 <= ||u-v||_2, while the Parseval identity distributes energy across low-pass and band-pass channels. The paper establishes that the resulting all-depth scattering representation is norm-preserving whenever widehat{phi}(0) != 0, without analyticity or frequency-covering assumptions. For f in the Sobolev space H^s(R^d), the unobserved depth-N residual obeys ||R_N f||_2 = O(N^{-min{s,1}/d}); equivalently, there is a constant C_f such that ||R_N f||_2 <= C_f N^{-min{s,1}/d}. Here d is spatial dimension, s is Sobolev smoothness, N is maximum path depth, and R_N is the collection of propagated coefficients below depth N. The rate is a safe polynomial stopping rule; no universal exponential rate is valid.

## Key formulas

- $$|\widehat{\phi}(\xi)|^{2}+\sum_{\lambda\in\Lambda}|\widehat{\psi}_{\lambda}(\xi)|^{2}=1,\qquad \widehat{\phi}(0)\neq 0.$$
- $$U[\emptyset]f=f,\qquad U[(p,\lambda)]f=\left|U[p]f*\psi_{\lambda}\right|,\qquad S[p]f=U[p]f*\phi.$$
- $$\left\|\,|u|-|v|\,\right\|_{2}\leq \|u-v\|_{2},\qquad \|R_{N}f\|_{2}\leq C_{f}N^{-\min\{s,1\}/d},\quad f\in H^{s}(\mathbb{R}^{d}).$$
- $$\mathcal{E}_{N}(f)=\sum_{|p|<N}\|S[p]f\|_{2}^{2}+\|R_{N}f\|_{2}^{2}=\|f\|_{2}^{2}.$$

## Implementation notes

Integrate the stem before the learned CNN or ViT patch embedding. For an input batch x with d=2, choose K complex or real convolution filters psi_lambda and one low-pass phi. First construct an unconstrained bank in the Fourier domain, then normalize every frequency bin by q(xi)=sqrt(|phi_hat(xi)|^2+sum_lambda|psi_hat_lambda(xi)|^2+epsilon), replacing each multiplier by its value divided by q; separately ensure phi_hat(0) is nonzero. Implement paths breadth-first: initialize paths with U_empty=x; for depth m=0,...,N-1, compute y_lambda=abs(conv_fft(U_path,psi_lambda)), append conv_fft(y_lambda,phi) to the feature tensor, and retain y_lambda only if its measured energy justifies expanding the path. Use the residual estimator r_N^2=max(0, ||x||_2^2-sum_{|p|<N}||S[p]x||_2^2), evaluated on a minibatch; stop expanding when r_N^2/||x||_2^2 is below a target such as 1%. The Parseval partition and modulus inequality come from the mathematics; the finite-grid FFT normalization, boundary handling, and the constant C_f are empirical engineering choices. Start with CIFAR-10, a 3-depth 8-channel-per-layer scattering stem, and compare against a same-FLOP learned ConvNet stem and a standard fixed wavelet scattering baseline. Measure energy drift, gradient norm variance, accuracy at equal parameter count, and accuracy versus depth. Success means near-zero energy-accounting error, no exploding gradients, and equal-or-better accuracy with fewer learned front-end parameters; a secondary signal is that depth chosen by the residual estimator avoids accuracy loss while reducing convolutional path count.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.