{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace a quadratic pairwise attention or graph aggregation kernel with a compact, translation-invariant indefinite kernel approximated by signed random Fourier features. The feature map preserves the kernel's negative spectral mass through a diagonal sign matrix, so the resulting linear-time aggregation can represent similarities that ordinary positive-definite random features cannot.",
 "formulas_latex": [
  "$$\\int_{\\mathbb{R}^{d}}f(\\boldsymbol{x})g(\\boldsymbol{x})d\\boldsymbol{x}=\\mathbb{E}[g(\\boldsymbol{X})],\\qquad \\frac{1}{M}\\sum_{m=1}^{M}g(\\boldsymbol{X}_m)$$",
  "$$K(\\boldsymbol{u})=\\int_{\\mathbb{R}^{d}}e^{i\\boldsymbol{\\omega}^{\\top}\\boldsymbol{u}}\\widehat K(\\boldsymbol{\\omega})d\\boldsymbol{\\omega}=C\\,\\mathbb{E}_{\\boldsymbol{\\Omega}\\sim q}\\left[s(\\boldsymbol{\\Omega})\\cos(\\boldsymbol{\\Omega}^{\\top}\\boldsymbol{u})\\right],\\quad q(\\boldsymbol{\\omega})=\\frac{|\\widehat K(\\boldsymbol{\\omega})|}{C},\\quad C=\\int|\\widehat K(\\boldsymbol{\\omega})|d\\boldsymbol{\\omega}$$",
  "$$\\widehat K_M(\\boldsymbol{x},\\boldsymbol{y})=\\frac{C}{M}\\sum_{m=1}^{M}s_m\\cos\\!\\left(\\boldsymbol{\\Omega}_m^{\\top}(\\boldsymbol{x}-\\boldsymbol{y})\\right)=\\frac{C}{M}z(\\boldsymbol{x})D z(\\boldsymbol{y})^{\\top}$$",
  "$$\\widetilde{V}=\\frac{C}{M}Z D\\left(Z^{\\top}V\\right),\\qquad Z_{i,2m-1}=\\cos(\\boldsymbol{\\Omega}_m^{\\top}\\boldsymbol{x}_i),\\quad Z_{i,2m}=\\sin(\\boldsymbol{\\Omega}_m^{\\top}\\boldsymbol{x}_i)$$"
 ],
 "id": 2820,
 "implementation": "(1) Integration point: implement a linear attention or graph-message-passing layer. Project each token or node to $x_i=W_xh_i$ and $y_j=W_yh_j$, optionally divide both by a learned bandwidth $\\ell$; use the signed radial interaction $K(x_i-y_j)$ instead of forming softmax logits. The value tensor is $V=W_vH$. Keep a separate positive normalization, such as $n_i=\\sum_j|\\widehat K_M(x_i-y_j)|+\\epsilon$, rather than dividing by a signed sum. (2) Pseudocode: sample $M$ frequencies $\\Omega_m\\sim q(\\omega)=|\\widehat K(\\omega)|/C$ and set $s_m=\\operatorname{sign}(\\widehat K(\\Omega_m))$; construct $Z_x=[\\cos(X\\Omega),\\sin(X\\Omega)]$ and $Z_y=[\\cos(Y\\Omega),\\sin(Y\\Omega)]$; compute $U=(C/M)Z_y^\\top V$, replace it by $DU$ with $D=\\operatorname{diag}(s_1,s_1,\\ldots,s_M,s_M)$, and return $Z_x(DU)$. For cross-attention, use distinct query and key matrices but the same sampled frequencies. (3) What comes from the mathematics: the signed measure, the $C/M$ Monte Carlo scaling, the duplicated cosine/sine factorization, and the sign insertion are fixed by the formulas. Estimate $C$ and sample $q$ numerically for the chosen kernel; for Kuttner-Golubov kernels $K(u)=(1-\\|u\\|^\\alpha)^\\beta\\mathbf{1}_{\\{\\|u\\|\\le1\\}}$, obtain $\\widehat K$ by numerical radial Fourier integration or a precomputed lookup table, then use rejection sampling with proposal $p_0$ and envelope $B$ accepting $\\omega$ with probability $|\\widehat K(\\omega)|/(Bp_0(\\omega))$. (4) First experiment: train a 6-layer, 8-head transformer on CIFAR-10 treated as 196 image patches, and compare standard softmax attention, ordinary positive RFF attention, and signed attention at equal feature width and measured FLOPs. Start with $M\\in\\{32,64,128\\}$ and the parabolic kernel. Measure validation accuracy, wall-clock time, memory, and approximation error against exact pairwise aggregation on a small batch. Success is lower memory and superlinear sequence-length scaling with no more than 1 percentage point accuracy loss; a stronger result is faster loss descent or improved accuracy from the compact negative lobes at equal FLOPs.",
 "math_summary": "The paper starts from the Monte Carlo identity $\\int f(\\boldsymbol{x})g(\\boldsymbol{x})d\\boldsymbol{x}=\\mathbb{E}[g(\\boldsymbol{X})]$ and estimator $M^{-1}\\sum_{m=1}^{M}g(\\boldsymbol{X}_m)$, then applies it to a signed Fourier density. Let $K:\\mathbb{R}^{d}\\to\\mathbb{R}$ be an even translation-invariant kernel, let $\\widehat K(\\boldsymbol{\\omega})$ be its Fourier transform under the convention $K(\\boldsymbol{u})=\\int_{\\mathbb{R}^{d}}e^{i\\boldsymbol{\\omega}^{\\top}\\boldsymbol{u}}\\widehat K(\\boldsymbol{\\omega})d\\boldsymbol{\\omega}$, and assume $\\int|\\widehat K(\\boldsymbol{\\omega})|d\\boldsymbol{\\omega}=C\u003c\\infty$. Define $q(\\boldsymbol{\\omega})=|\\widehat K(\\boldsymbol{\\omega})|/C$ and $s(\\boldsymbol{\\omega})=\\operatorname{sign}(\\widehat K(\\boldsymbol{\\omega}))\\in\\{-1,+1\\}$. If $\\boldsymbol{\\Omega}_m\\sim q$ independently, then $K(\\boldsymbol{u})=C\\mathbb{E}[s(\\boldsymbol{\\Omega})\\cos(\\boldsymbol{\\Omega}^{\\top}\\boldsymbol{u})]$ for real even kernels. Using $\\cos(a-b)=\\cos a\\cos b+\\sin a\\sin b$, define $z(\\boldsymbol{x})=[\\cos(\\boldsymbol{\\Omega}_1^{\\top}\\boldsymbol{x}),\\sin(\\boldsymbol{\\Omega}_1^{\\top}\\boldsymbol{x}),\\ldots,\\cos(\\boldsymbol{\\Omega}_M^{\\top}\\boldsymbol{x}),\\sin(\\boldsymbol{\\Omega}_M^{\\top}\\boldsymbol{x})]$. Then $K(\\boldsymbol{x}-\\boldsymbol{y})\\approx (C/M)z(\\boldsymbol{x})D z(\\boldsymbol{y})^{\\top}$, where $D$ is diagonal and repeats $s(\\boldsymbol{\\Omega}_m)$ twice. For values $V\\in\\mathbb{R}^{N\\times r}$, all-pairs aggregation $AV$ with $A_{ij}=K(x_i-x_j)$ becomes $(C/M)Z D(Z^{\\top}V)$ instead of an $N\\times N$ product.",
 "math_tags": [
  "probability",
  "harmonic-analysis",
  "linear-algebra",
  "measure-theory"
 ],
 "ml_areas": [
  "attention",
  "graph-nn",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2608.29265",
  "arxiv_url": "https://arxiv.org/abs/2608.29265",
  "summary_what_math_gives_to_ml": "The paper provides a constructive Monte Carlo factorization for translation-invariant kernels whose Fourier transform is signed rather than nonnegative. This is useful beyond KDE because many neural architectures implicitly require positive-definite random-feature kernels, excluding compact-support similarities and other indefinite interactions. The transferable asset is the signed spectral measure: sample frequencies from the absolute spectral density and carry a separate sign into the feature contraction, yielding linear-time approximations to indefinite pairwise aggregation. A promising neural use is replacing quadratic attention or graph message passing with a compact radial signed-kernel interaction, while explicitly testing whether signed cancellation improves locality or expressivity without destabilizing normalization.",
  "title": "Signed random Fourier features for fast density estimation with indefinite kernels",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "speedup",
  "accuracy",
  "scalability"
 ],
 "title": "Signed spectral attention",
 "url": "https://synthcore.org/idea/2820/signed-spectral-attention",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}