{
 "artifacts": null,
 "category": "regularization",
 "description": "Replace a dense Gaussian random-feature layer by a sparse Poisson-chaos layer with the same learned kernel, and penalize the layer's normalized fourth add-one energy. The penalty suppresses rare single-atom jumps, so the sparse layer approaches the Gaussian same-kernel representation in Wasserstein distance while preserving computational advantages from sparse Poisson sampling.",
 "formulas_latex": [
  "$$Z_{\\nu}=2X_{\\nu/2,1}-\\nu,$$",
  "$$F_n\\xrightarrow{\\mathrm d}N(0,1)\\quad\\Longleftrightarrow\\quad \\mathbb E[F_n^4]\\longrightarrow 3,$$",
  "$$D_xF=F(\\eta+\\delta_x)-F(\\eta),\\qquad \\mathcal E_4(F)=\\int \\mathbb E\\!\\left[|D_xF|^4\\right]\\,\\mu(dx),$$",
  "$$\\mathcal L_{\\mathrm{total}}=\\mathcal L_{\\mathrm{task}}+\\lambda\\,\\frac{\\widehat{\\mathcal E}_4(F)}{(\\widehat{\\operatorname{Var}}(F)+\\varepsilon)^2}+\\beta\\left(\\widehat{\\operatorname{Var}}(F)-1\\right)^2.$$"
 ],
 "id": 3039,
 "implementation": "(1) Integration point: implement this as a replacement for a Gaussian random-feature projection or as an optional sparse MLP or embedding projection. Discretize the feature domain into \\(m\\) atoms with Poisson rates \\(\\lambda_j\\). For an MVP, use order \\(q=2\\), a symmetric kernel matrix \\(K\\in\\mathbb R^{m\\times m}\\) with zero diagonal, and centered counts \\(\\xi_j=(N_j-\\lambda_j)/\\sqrt{\\lambda_j}\\), where \\(N_j\\sim\\operatorname{Poisson}(\\lambda_j)\\). For input \\(h\\), use a learned input-dependent kernel \\(K(h)\\), or first project \\(h\\) to coefficients \\(a_j(h)\\); form \\(F(h)=\\sum_{j\\ne k}K_{jk}(h)\\xi_j\\xi_k\\). Form the Gaussian control feature \\(G(h)=\\sum_{j\\ne k}K_{jk}(h)z_jz_k\\), with \\(z_j\\sim N(0,1)\\), using the same kernel. (2) Pseudocode: sample counts \\(N\\), compute \\(\\xi\\), and compute \\(F\\). For each selected atom \\(r\\), evaluate an add-one value by replacing \\(N_r\\leftarrow N_r+1\\), recomputing \\(F^{(+r)}\\), and setting \\(D_rF=F^{(+r)}-F\\). Accumulate the minibatch importance-sampling estimate \\(\\widehat{\\mathcal E}_4=\\frac1B\\sum_{b=1}^B\\sum_{r\\in S_b}\\mu_r|D_rF_b|^4/p_r\\), where \\(S_b\\) is a sampled atom subset, \\(p_r\\) is its inclusion probability, and \\(\\mu_r\\) is the atom intensity. Add the normalized penalty and variance penalty to the task loss, then backpropagate. (3) The paper-derived object is the fourth add-one energy and its vanishing-energy invariance interpretation. The minibatch variance, atom subsampling, inclusion probabilities, and differentiable finite-difference estimator are engineering choices. Use common random numbers when comparing Poisson and Gaussian features so both use identical kernels and inputs. (4) First cheap experiment: train a two-layer MLP on MNIST with a 512-feature Gaussian random projection as baseline. Compare it with a Poisson order-2 layer at equal expected active-feature count, both without regularization and with the add-one penalty. Track validation accuracy, gradient-norm variance, activation kurtosis, empirical Wasserstein distance between Poisson and Gaussian outputs, and wall-clock cost. Also test a rare-jump regime with small atom rates; the hypothesis is that add-one regularization improves stability and accuracy there, reducing activation tails and Poisson-versus-Gaussian discrepancy at equal sparsity. A negative result is equally clear: if the penalty does not improve accuracy or gradient stability at fixed expected active count, discard the method.",
 "math_summary": "For a Poisson process \\(\\eta\\) with intensity measure \\(\\mu\\), let \\(F=I_q(f)\\) be a centered multiple Poisson integral of fixed order \\(q\\), where \\(f\\) is a symmetric kernel and \\(I_q\\) is the compensated \\(q\\)-fold stochastic integral. The add-one derivative is \\(D_xF=F(\\eta+\\delta_x)-F(\\eta)\\), where \\(\\delta_x\\) adds one atom at \\(x\\). Define the fourth add-one energy by \\(\\mathcal E_4(F)=\\int \\mathbb E[|D_xF|^4]\\,\\mu(dx)\\). The paper's Poisson-Gaussian invariance principle states that, for variance-normalized fixed-order chaoses, \\(\\mathcal E_4(F_n)\\to0\\) implies convergence of the Poisson integral \\(I_q^{\\eta}(f_n)\\) and the Gaussian same-kernel integral \\(I_q^{G}(f_n)\\) in smooth-test and Wasserstein distance. The Gaussian reference criterion is \\(F_n\\xrightarrow{\\mathrm d}N(0,1)\\Longleftrightarrow\\mathbb E[F_n^4]\\to3\\) for unit-variance fixed Gaussian chaos. In a neural layer, use \\(\\mathcal E_4(F)/\\operatorname{Var}(F)^2\\) as a regularizer: it penalizes sensitivity to one extra sampled atom, which controls rare-jump behavior more directly than ordinary kurtosis.",
 "math_tags": [
  "stochastic-processes",
  "probability",
  "statistics",
  "information-theory"
 ],
 "ml_areas": [
  "mlp",
  "embedding",
  "regularization",
  "initialization"
 ],
 "paper": {
  "arxiv_id": "2609.02136",
  "arxiv_url": "https://arxiv.org/abs/2609.02136",
  "summary_what_math_gives_to_ml": "The paper identifies the fourth add-one energy of a Poisson functional as an intrinsic Lindeberg quantity: when this energy vanishes after variance normalization, a fixed-order Poisson chaos and the Gaussian chaos built from the same kernel become indistinguishable in distribution. This is stronger and more structurally informative than matching a few ordinary moments, because rare jumps can preserve low-order moments while still preventing Gaussian behavior. A transferable use is to build sparse Poisson random-feature or noise layers whose kernels are explicitly regularized toward low add-one energy, obtaining Gaussian-like representations while retaining discrete or sparse sampling. The criterion can also diagnose when Gaussian initialization or Gaussian dropout is a faithful replacement for a sparse Poisson mechanism.",
  "title": "Gamma approximation and Poisson--Gaussian invariance principle on Poisson chaos",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 8,
  "usefulness": 6
 },
 "solves": [
  "accuracy",
  "stability",
  "sample-efficiency"
 ],
 "title": "Lindeberg-Regularized Poisson Random Features",
 "url": "https://synthcore.org/idea/3039/lindeberg-regularized-poisson-random-features",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}