Local tent-space wavelet sparsifier

(1) Integration point: apply a 2-D Haar or biorthogonal wavelet transform independently to each channel of an intermediate feature tensor F with shape C by H by W. Treat low-frequency coefficients on unit spatial tiles as beta_k and all detail coefficients at dyadic squares Q as alpha(Q). For a transformer, apply the transform to a reshaped image-token grid before one attention block. (2) Pseudocode: compute alpha,beta = wavelet(F); initialize e[x]=0; for every father coefficient beta[k], add beta[k]^2 to e[x] for pixels x in R_k; for every mother coefficient alpha[Q], add alpha[Q]^2 divided by area(Q) to e[x] for pixels x in R(Q); compute S[x]=sqrt(e[x]+epsilon); set regularizer R=sum_x S[x]^p; optimize task_loss plus lambda times R. Start with p=0.5, R_k=Q_k, and R(Q)=Q, or use fixed central half-cubes to imitate the positive-measure subsets. (3) Since p<1 is nonconvex and has a singular derivative at zero, use an iteratively reweighted quadratic surrogate. Every T steps set w[x]=(S[x]^2+epsilon)^(p/2-1), then optimize the surrogate sum_x w[x]e[x] for the next T steps. The wavelet transform, cube areas, and square-function expression are determined by the paper; lambda, p, epsilon, update period, and subset geometry are empirical. (4) First experiment: train ResNet-18 on CIFAR-10 with fixed Haar regularization at one intermediate resolution. Compare no regularizer, coefficient-wise l_1, spatial group-lasso, and this p=0.5 penalty at matched clean accuracy. Measure the proportion of eliminated wavelet groups, activation memory after thresholding, validation accuracy, and robustness to additive high-frequency noise. Success means lower active-feature memory or better high-frequency robustness at equal accuracy, with whole localized coefficient groups disappearing rather than only individual coefficients shrinking.