{
 "artifacts": null,
 "category": "architecture",
 "description": "Partition activations into dyadic magnitude bands and allocate sparse connectivity separately to heavy and diffuse coordinates. Protect high-magnitude coordinates with more reliable connections while using randomized flat connectivity for the many small coordinates, keeping the total number of nonzeros fixed.",
 "formulas_latex": [
  "$$I_j(x):=\\{i\\in[n]:2^{-j}\u003c\\left|x_i\\right|\\leq 2^{-j+1}\\}.$$",
  "$$\\left\\lVert\\Pi x\\right\\rVert_2\\leq\\left\\lVert\\mathcal{T}_L^{\\rm can}(\\Pi,x)x\\right\\rVert_2+\\left\\lVert\\mathcal{F}_L^{\\rm can}(\\Pi,x)x\\right\\rVert_2.$$",
  "$$\\left\\lVert\\Pi U_V\\right\\rVert\\leq 2\\sup_{z\\in\\mathcal{N}_V}\\left\\lVert\\mathcal{T}_L^{\\rm can}(\\Pi,z)z\\right\\rVert_2+2\\sup_{z\\in\\mathcal{N}_V}\\left\\lVert\\mathcal{F}_L^{\\rm can}(\\Pi,z)z\\right\\rVert_2.$$",
  "$$\\widetilde W_{ij}=\\frac{b_{ij}\\xi_{ij}}{\\sqrt{k p_{j_i}}},\\qquad p_j=\\min\\left\\{1,p_0 2^{2j}\\right\\}.$$"
 ],
 "id": 2921,
 "implementation": "Apply the mechanism to a token-mixing or MLP projection y = W h, with h in R^n and W in R^{k x n}. During calibration, compute a_i = |h_i| + 10^{-8} and assign each coordinate to a dyadic band j_i = floor(-log_2(a_i)); clip j_i to [-8,8]. Define heavy coordinates as the largest five percent by magnitude, or as coordinates above a fixed percentile threshold, and define all others as flat. Give every output row a fixed heavy-coordinate budget d_H and a flat-coordinate budget d_F, with d_H + d_F equal to the nonzero budget of the unstructured sparse baseline. Freeze the resulting mask after calibration for deployment. Use W_ij = b_ij xi_ij/sqrt(k p_{j_i}), where p_j = min(1, p_0 2^{2j}) after choosing the sign convention so larger j corresponds to smaller magnitudes; alternatively estimate p_j from calibration by making p_j proportional to the inverse conditional second moment E[a_i^2 | j_i = j], clipped to [p_F,1]. Pseudocode: compute magnitude bands; select heavy coordinates; sample or assign protected heavy connections; sample flat connections; apply per-band scaling; compute y. The paper contributes the level-set and Tall-plus-Flat decomposition; density schedules and thresholds are engineering choices. First test a width-512 Transformer on WikiText-2 or a small ViT on CIFAR-100 against an unstructured mask with identical parameters, nonzeros, optimizer, and measured FLOPs. Pre-register a 20 percent reduction in the 99.9th-percentile amplification ratio ||Wh||/||h||, lower variance of the heavy contribution ||W h_heavy||/||Wh||, and a smaller loss increase when flat coordinates are ablated instead of heavy coordinates. The mechanism ablation is a same-density global random mask with global 1/sqrt(kp) scaling and no level-set allocation. Falsify the idea if frozen level-set masks do not reduce amplification tails by at least 10 percent or if the result vanishes against a magnitude-aware but non-dyadic baseline.",
 "math_summary": "For x in R^n, the paper defines the dyadic level set I_j(x) := {i in [n] : 2^{-j} \u003c |x_i| \u003c= 2^{-j+1}}. It splits the sparse action into a Tall part and a Flat part and uses the deterministic inequality ||Pi x||_2 \u003c= ||T_L^can(Pi,x)x||_2 + ||F_L^can(Pi,x)x||_2. For a 1/2-net N_V contained in V intersected with the unit sphere, it further gives ||Pi U_V|| \u003c= 2 sup_{z in N_V} ||T_L^can(Pi,z)z||_2 + 2 sup_{z in N_V} ||F_L^can(Pi,z)z||_2. The surrounding discussion identifies heavy columns and coordinates with the Tall contribution, handled by row concentration, and light coordinates and couples with the Flat contribution, controlled through entropy of level-set configurations. The neural adaptation freezes a level-set-aware mask after calibration, avoiding a fully input-dependent mask during training. The exact theorem does not apply to adaptive neural masks, so this is a falsifiable architectural heuristic rather than a direct guarantee.",
 "math_tags": [
  "random-matrix",
  "probability",
  "linear-algebra",
  "combinatorics"
 ],
 "ml_areas": [
  "mlp",
  "transformer",
  "regularization",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2607.23017",
  "arxiv_url": "https://arxiv.org/abs/2607.23017",
  "summary_what_math_gives_to_ml": "The paper develops a constructive method for controlling sparse random operators on low-dimensional subspaces by decomposing vectors into dyadic coordinate level sets and separating heavy (Tall) and diffuse (Flat) contributions. The most transferable asset is the effective scale r_* = max{r, log(k)/p}, which identifies when sparsity rather than intrinsic dimension controls fluctuations, together with the normalization ||Pi U_V|| = O(sqrt(kp)). A practical neural-network transfer is a spectrally calibrated sparse projection or token-mixing layer whose density and initialization are chosen from the activation rank and sparsity. A second experiment can use magnitude bands to allocate connectivity unequally, protecting large coordinates while using randomized connectivity for diffuse coordinates.",
  "title": "Level-set entropy and sparse randomized embeddings",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "accuracy",
  "memory"
 ],
 "title": "Level-Set Balanced Sparse Mixer",
 "url": "https://synthcore.org/idea/2921/level-set-balanced-sparse-mixer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}