KS-Balanced Spectral Residual Block

1. Integration point: use the module as a residual block for image features, token sequences, or latent states. Interpret the spatial or sequence axis as a periodic grid, apply an FFT independently to each channel, and use a small pointwise MLP \(G_\theta\) for nonlinear feature mixing. In a transformer, insert it in parallel with the feed-forward sublayer; in an MLP or SSM, place it between normalized residual blocks. 2. Pseudocode: initialize \(a=\operatorname{softplus}(a_0)\) and \(b=\operatorname{softplus}(b_0)\); compute \(H=\operatorname{FFT}(h)\), \(L_k=a|k|^2-b|k|^4\), and \(L_{\max}=\max_k L_k\); set \(\eta_{\mathrm{eff}}=\min(\eta,r_{\max}/\max(L_{\max},\epsilon))\); return \(h+\eta_{\mathrm{eff}}\operatorname{IFFT}(L_kH)+\eta_{\mathrm{eff}}G_\theta(h,\operatorname{IFFT}(|k|H))\). 3. Computed from the paper: the second/fourth derivative competition, the finite cutoff, and the need for infrared amplification balanced by ultraviolet damping. Estimated empirically: \(a\), \(b\), nonlinear gain, and \(r_{\max}\). 4. First cheap experiment: train a length-256 1D sequence classifier and autoregressive copy/reversal model, comparing a standard residual MLP, a purely contractive Fourier block, and this KS-balanced block at equal parameter count. Sweep \(a/b\) and depth while logging per-layer Fourier gains. The quantitative prediction is a spectral transition near \(k_c^{\mathrm{NN}}=\sqrt{a/b}\): modes below the cutoff should have positive gain and modes above it negative gain. For explicit Euler updates, divergence should begin when \(\eta L_{\max}\) approaches order one; adaptive gain control should remove this boundary while retaining stronger long-range correlations than the contractive baseline.