{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace a dense Gaussian or learned projection from dimension N to m with a normalized partial circulant projection generated by a single Gaussian vector. For K-sparse hidden states, the restricted-isometry guarantee predicts approximate norm preservation while reducing stored projection parameters from O(mN) to O(N). The projection can be evaluated with an FFT and should be combined with explicit top-k gating so that the sparse-input assumption is enforced.",
 "formulas_latex": [
  "$$A=\\frac{1}{\\sqrt{m}}P_{\\Omega}C_g,\\qquad (C_gx)_i=\\sum_{j=0}^{N-1}g_{(i-j)\\bmod N}x_j,\\qquad g\\sim\\mathcal N(0,I_N),\\quad |\\Omega|=m.$$",
  "$$(1-\\delta)\\|x\\|_2^2\\leq \\|Ax\\|_2^2\\leq(1+\\delta)\\|x\\|_2^2\\qquad\\text{for every }x\\text{ with }\\|x\\|_0\\leq K.$$",
  "$$m\\geq C\\delta^{-2}K\\max\\left\\{\\log^2(eK)\\log(2N)\\log(em),\\,\\log(2/\\eta)\\right\\}.$$",
  "$$\\text{coarse-scale entropy factor: }\\sqrt{\\log(em)}\\text{ rather than }\\sqrt{\\log N},\\qquad\\text{so the squared factor is }\\log(em).$$"
 ],
 "id": 1500,
 "implementation": "Integrate this operator as a replacement for the first linear projection in a sparse MLP block, sparse autoencoder, or token-mixing layer. The input tensor should have shape [batch,N]. Choose a fixed row set Omega of size m and store one vector g of length N. First enforce sparsity with a top-k mask: `support = topk_abs(x,K); xs = x * support`. Compute the circular convolution using the FFT, then subsample and normalize: `z = ifft(fft(xs) * fft(g)).real; y = z[...,Omega] / sqrt(m)`. Apply bias, normalization, and the nonlinearity after this operation. Start with frozen Gaussian g to test the mathematical effect, then compare a version where g is trainable. Use the theorem as a width heuristic: `m = ceil(C_hat * delta**-2 * K * max(log(e*K)**2 * log(2*N) * log(e*m), log(2/eta)))`, sweeping C_hat over {1,2,4,8} because the universal constant is unspecified. The paper supplies the bound; estimate actual distortion empirically by generating many random K-sparse vectors x and recording `abs(||Ax||_2^2 / ||x||_2^2 - 1)`, including the maximum and 99th percentile. Begin with a 2-layer MLP on MNIST or CIFAR-10, N=1024, m in {256,512,1024}, and K in {32,64,128}. Compare against dense Gaussian and dense learned projections at equal output width. Measure validation accuracy, loss versus optimizer steps and wall-clock time, parameter memory, FFT cost, and distortion. Success is at least 4x lower projection parameter memory with no more than 1 percentage point accuracy loss and lower norm distortion than an unstructured random projection at the same parameter budget.",
 "math_summary": "Let N be the input dimension, m the number of output measurements, K the sparsity level, and g=(g_0,...,g_{N-1}) sampled from the standard Gaussian distribution N(0,I_N). Define circular convolution C_g by (C_g x)_i = sum_{j=0}^{N-1} g_{(i-j) mod N} x_j. Let Omega be any fixed subset of Z_N with |Omega|=m and P_Omega select those rows. The normalized partial circulant matrix is A=(1/sqrt(m))P_Omega C_g. Its restricted-isometry constant of order K is the smallest delta_K such that (1-delta_K)||x||_2^2 \u003c= ||Ax||_2^2 \u003c= (1+delta_K)||x||_2^2 for every vector x with at most K nonzero entries. The paper proves that a universal constant C exists such that delta_K \u003c= delta with probability at least 1-eta if m \u003e= C delta^{-2} K max{log^2(eK) log(2N) log(em), log(2/eta)}. The proof's transferable refinement is a noncommutative Khintchine inequality and Schatten-moment estimate whose coarse-scale Rademacher contribution is sqrt(log(em)) rather than sqrt(log N); Maurey's entropy argument squares this contribution, yielding log(em). In the neural layer, K is the enforced top-k support, N is input width, m is output width, delta is the desired distortion, and eta is the failure probability.",
 "math_tags": [
  "random-matrix",
  "linear-algebra",
  "harmonic-analysis",
  "probability"
 ],
 "ml_areas": [
  "mlp",
  "cnn",
  "initialization",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2607.27676",
  "arxiv_url": "https://arxiv.org/abs/2607.27676",
  "summary_what_math_gives_to_ml": "The paper gives a sharper high-probability restricted-isometry guarantee for subsampled circular convolutions generated by one Gaussian vector. The transferable asset is a parameter-efficient structured projection that preserves norms of sparse inputs, with an FFT implementation and a width requirement improved from a second ambient-dimension logarithm to a measurement-dependent logarithm. This is most promising for neural layers whose inputs are explicitly top-k sparse or otherwise sparse-coded. The theorem supplies a principled measurement-width heuristic and a direct empirical test through norm-distortion measurements.",
  "title": "Improved RIP Bounds for Gaussian Partial Circulant Matrices",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "memory",
  "speedup",
  "stability"
 ],
 "title": "RIP-Circulant Sparse Projection Layer",
 "url": "https://synthcore.org/idea/1500/rip-circulant-sparse-projection-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}