Beats tuned baseline
2026
Private spectral whitening front-end
Estimate the temporal spectrum of each sequence channel using a locally private procedure, then apply a regularized inverse-square-root spectral filter before the sequence enters attention or an SSM. The filter removes predictable low-frequency or narrow-band redundancy while avoiding unstable amplification at frequencies where the private estimate is small.
Paper: On the privacy cost for dependent Gaussian data: spectral density estimation under local differential privacy
arXiv:2608.24847