Prolate Energy-Preserving Bottleneck

Integrate this at the input of an expensive sequence module. For a tensor X with shape batch by sequence length T by channels D, apply the same orthonormal DPSS matrix U along the sequence axis before attention, a convolutional stack, or an MLP. Choose an effective normalized bandwidth W in (0,0.5), construct the symmetric concentration matrix C using the sinc formula above, and compute its eigenvectors with a symmetric eigensolver. Sort eigenvalues descending and retain r modes. Set delta to the maximum acceptable discarded concentration, compute barL=log((1-delta)/delta), and use r_asym=c + barL/pi^2 * log(4*pi^2*c/barL) as an initial rank estimate; because discrete normalization differs from the continuous theorem, clip r_asym to [1,T] and verify it using the empirical eigenvalues. Pseudocode: construct C; eigvals,eigvecs=eigh(C); sort descending; r=ceil(r_asym); Z=einsum('btd,tr->brd',X,U[:,:r]); Y=expensive_block(Z); output=einsum('brd,tr->btd',Y,U[:,:r].T), optionally adding a residual bypass for discarded coordinates. The theorem supplies the rank schedule; the eigendecomposition and actual reconstruction error are computed numerically. Start with a small Transformer on WikiText-103 or Long Range Arena at T=1024, comparing full attention, Fourier truncation, random orthogonal projection, and a learned linear bottleneck. Measure validation perplexity, reconstruction error, peak activation memory, and attention FLOPs. Success means at least 25 percent lower attention memory or FLOPs with no more than 1 percent relative perplexity degradation, and a better rank-versus-error curve than Fourier truncation on locally concentrated signals.