Level-Set Balanced Sparse Mixer

Apply the mechanism to a token-mixing or MLP projection y = W h, with h in R^n and W in R^{k x n}. During calibration, compute a_i = |h_i| + 10^{-8} and assign each coordinate to a dyadic band j_i = floor(-log_2(a_i)); clip j_i to [-8,8]. Define heavy coordinates as the largest five percent by magnitude, or as coordinates above a fixed percentile threshold, and define all others as flat. Give every output row a fixed heavy-coordinate budget d_H and a flat-coordinate budget d_F, with d_H + d_F equal to the nonzero budget of the unstructured sparse baseline. Freeze the resulting mask after calibration for deployment. Use W_ij = b_ij xi_ij/sqrt(k p_{j_i}), where p_j = min(1, p_0 2^{2j}) after choosing the sign convention so larger j corresponds to smaller magnitudes; alternatively estimate p_j from calibration by making p_j proportional to the inverse conditional second moment E[a_i^2 | j_i = j], clipped to [p_F,1]. Pseudocode: compute magnitude bands; select heavy coordinates; sample or assign protected heavy connections; sample flat connections; apply per-band scaling; compute y. The paper contributes the level-set and Tall-plus-Flat decomposition; density schedules and thresholds are engineering choices. First test a width-512 Transformer on WikiText-2 or a small ViT on CIFAR-100 against an unstructured mask with identical parameters, nonzeros, optimizer, and measured FLOPs. Pre-register a 20 percent reduction in the 99.9th-percentile amplification ratio ||Wh||/||h||, lower variance of the heavy contribution ||W h_heavy||/||Wh||, and a smaller loss increase when flat coordinates are ablated instead of heavy coordinates. The mechanism ablation is a same-density global random mask with global 1/sqrt(kp) scaling and no level-set allocation. Falsify the idea if frozen level-set masks do not reduce amplification tails by at least 10 percent or if the result vanishes against a magnitude-aware but non-dyadic baseline.