Parseval Scattering Stem with Certified Depth
Implementation & benchmark of arXiv:2608.18064 — Universal admissibility for scattering transforms
Source paper: Universal admissibility for scattering transforms arXiv:2608.18064 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace the first several convolutional blocks of a small image model with a finite-depth convolution-modulus scattering stem built from a Parseval filter bank. Enforce exact energy accounting and use the paper's polynomial residual law to choose the smallest depth that captures the desired fraction of input energy, avoiding unstable or redundant deep scattering paths.
Formulas
Mathematical statement
The filter bank consists of a low-pass filter phi and band-pass filters psi_lambda whose Fourier multipliers satisfy the Parseval partition |widehat{phi}(xi)|^2 + sum_lambda |widehat{psi_lambda}(xi)|^2 = 1 for every frequency xi in R^d. For a path p=(lambda_1,...,lambda_m), define U[emptyset]f=f and U[(p,lambda)]f = |U[p]f * psi_lambda|, where * is convolution and |.| is pointwise modulus; the scattering coefficient is S[p]f = U[p]f * phi. The modulus is nonexpansive, || |u|-|v| ||_2 <= ||u-v||_2, while the Parseval identity distributes energy across low-pass and band-pass channels. The paper establishes that the resulting all-depth scattering representation is norm-preserving whenever widehat{phi}(0) != 0, without analyticity or frequency-covering assumptions. For f in the Sobolev space H^s(R^d), the unobserved depth-N residual obeys ||R_N f||_2 = O(N^{-min{s,1}/d}); equivalently, there is a constant C_f such that ||R_N f||_2 <= C_f N^{-min{s,1}/d}. Here d is spatial dimension, s is Sobolev smoothness, N is maximum path depth, and R_N is the collection of propagated coefficients below depth N. The rate is a safe polynomial stopping rule; no universal exponential rate is valid.
Implementation notes
Integrate the stem before the learned CNN or ViT patch embedding. For an input batch x with d=2, choose K complex or real convolution filters psi_lambda and one low-pass phi. First construct an unconstrained bank in the Fourier domain, then normalize every frequency bin by q(xi)=sqrt(|phi_hat(xi)|^2+sum_lambda|psi_hat_lambda(xi)|^2+epsilon), replacing each multiplier by its value divided by q; separately ensure phi_hat(0) is nonzero. Implement paths breadth-first: initialize paths with U_empty=x; for depth m=0,...,N-1, compute y_lambda=abs(conv_fft(U_path,psi_lambda)), append conv_fft(y_lambda,phi) to the feature tensor, and retain y_lambda only if its measured energy justifies expanding the path. Use the residual estimator r_N^2=max(0, ||x||_2^2-sum_{|p|<N}||S[p]x||_2^2), evaluated on a minibatch; stop expanding when r_N^2/||x||_2^2 is below a target such as 1%. The Parseval partition and modulus inequality come from the mathematics; the finite-grid FFT normalization, boundary handling, and the constant C_f are empirical engineering choices. Start with CIFAR-10, a 3-depth 8-channel-per-layer scattering stem, and compare against a same-FLOP learned ConvNet stem and a standard fixed wavelet scattering baseline. Measure energy drift, gradient norm variance, accuracy at equal parameter count, and accuracy versus depth. Success means near-zero energy-accounting error, no exploding gradients, and equal-or-better accuracy with fewer learned front-end parameters; a secondary signal is that depth chosen by the residual estimator avoids accuracy loss while reducing convolutional path count.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.