△ Mechanism confirmed, baseline not beaten 2026

Utility-Weighted Left-Edge Quantization

Implementation & benchmark of arXiv:2609.03745 — A Quantization Problem Posed by Adaptive Streaming

Usefulness7/10

Difficulty4/10

Novelty7/10

Memory Memory Accuracy Speedup Quantization Transformer Inference speedup Probability Approximation theory Measure theory Optimization

Source paper: A Quantization Problem Posed by Adaptive Streaming arXiv:2609.03745 ⓘ · analyzed Sep 4, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace MSE-calibrated scalar quantization with a conservative left-edge quantizer whose thresholds are denser where activation probability and task utility slope are both high. For a monotone utility function, this should preserve high-impact activation regions better than uniform or MSE-optimal bins at the same number of codes, while retaining an explicit rate-versus-quality design rule.

Formulas

$$i_B(B)=j\;\Longleftrightarrow\;R_j\leq B<R_{j+1},\qquad R_0=0,\quad R_{n+1}=\infty$$

$$Q(R)\in[0,1],\qquad Q(0)=0,\qquad Q(\infty)=1,\qquad Q'(R)>0$$

$$n\,\mathcal{G}_n^{*}\to C_w=\frac{1}{2}\left(\int\sqrt{p(x)Q'(x)}\,dx\right)^2$$

$$\hat{x}=R_j\ \text{for }x\in[R_j,R_{j+1}),\qquad \int_0^{R_j}\sqrt{p(x)Q'(x)}\,dx\approx \frac{j}{n}\int_0^{\infty}\sqrt{p(x)Q'(x)}\,dx$$

Mathematical statement

The paper models a monotone quality curve $Q(R)\in[0,1]$, with $Q(0)=0$, $Q(\infty)=1$, and strictly increasing $C^1$ behavior. A threshold ladder $0=R_0<R_1<\cdots<R_n<R_{n+1}=\infty$ uses the left-edge reproduction rule $i_B(B)=j\Longleftrightarrow R_j\le B<R_{j+1}$, so an input $B$ in cell $[R_j,R_{j+1})$ is represented by $R_j$. Let $p(x)$ be the empirical density of a nonnegative activation or weight magnitude $x$, and let $Q(x)$ be a monotone task-utility curve: the ideal utility at value $x$, while the quantized utility is $Q(R_j)$. The asymptotic quality-gap theorem states $\mathcal{G}_n^*=\Theta(1/n)$ and $n\mathcal{G}_n^*\to C_w$, where $C_w=\frac{1}{2}(\int\sqrt{p(x)Q'(x)}\,dx)^2$. The optimal threshold density is proportional to $\sqrt{p(x)Q'(x)}$, unlike the classical squared-error rule proportional to $p(x)^{1/3}$. For a finite codebook, define $g(x)=\sqrt{p(x)Q'(x)}$, $Z=\int g(x)dx$, and place thresholds so that $\int_0^{R_j}g(x)dx\approx jZ/n$. The left-edge rule is conservative: $\hat{x}=R_j\le x$, avoiding upward rounding when overestimation is harmful.

Implementation notes

Integrate this at the scalar activation-quantization point, initially for nonnegative post-ReLU activations or absolute values of weights in one Transformer MLP block. During calibration, collect a histogram $\{(x_k,c_k)\}$ of values in the chosen tensor, normalize it to an empirical density $p_k=c_k/\sum_l c_l$, and estimate a monotone utility curve $Q$. The cheapest first utility is $Q(x)=\log(1+x/s)/\log(1+x_{\max}/s)$, where $s$ is the tensor median; a stronger version estimates $Q'(x)$ by measuring validation-loss increase after selectively attenuating values in each histogram bin. Smooth the nonnegative weights $g_k=\sqrt{p_kQ'_k+\epsilon}$, compute $G_k=\sum_{l\le k}g_l$, and choose $n-1$ internal thresholds at cumulative targets $jG_K/n$. At inference, locate the largest threshold $R_j\le x$ and emit the left edge $R_j$, using integer code indices and a per-tensor scale/table. Pseudocode: hist=calibrate(x); p=normalize(hist); Qprime=utility_slope(hist); g=sqrt(p*Qprime+eps); G=cumsum(g); thresholds=invert_cdf(G,[1/n,...,(n-1)/n]); codes=searchsorted(thresholds,x)-1; xhat=thresholds[codes]. The paper-derived quantities are the $\sqrt{pQ'}$ allocation and $C_w=0.5(\sum_k\sqrt{p_kQ'_k}\Delta x)^2$; density and utility slope are estimated empirically. First test on a small 6-layer Transformer language model with INT4 activation quantization on WikiText-2, comparing uniform, percentile, and MSE Lloyd-Max quantizers at equal 4-bit memory and kernel cost. Measure validation perplexity, calibration loss, saturation rate, and throughput. Success is lower perplexity at the same bitwidth, or equal perplexity with fewer codes; also verify the predicted diminishing-return curve $\mathcal{G}_n\approx C_w/n$ for $n\in\{4,8,16,32\}$.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built and numerically verified Utility-Weighted Left-Edge Quantization in a self-contained toy experiment. The prescribed g=sqrt(pQ') allocation equalized per-bin g-mass with CV 1.9e-9, and the weighted utility gap decreased monotonically with n while n·gap approached the predicted Cw=0.4143, reaching 0.4092 at 64 codes. It consistently beat uniform left-edge quantization: at 64 codes the gap was 0.00639 versus 0.02040, while upward rounding remained exactly zero.

Agent confidence: 8/10

Baseline: Uniform left-edge utility gaps for n={4,8,16,32,64}: {0.24507, 0.15027, 0.08027, 0.04072, 0.02040}; n·gap at 64 was 1.3053.
Idea: Utility-weighted left-edge utility gaps for n={4,8,16,32,64}: {0.09325, 0.04854, 0.02488, 0.01264, 0.00639}; n·gap at 64 was 0.4092 versus theoretical Cw=0.4143; upward-rounding fraction was 0.0 for every codebook.

Limitations: This is a bounded synthetic exponential-distribution experiment, not the proposed Transformer/WikiText-2 validation. The utility curve is hand-selected logarithmic rather than estimated from task-loss perturbations, and throughput, memory, kernel cost, calibration loss, and end-to-end perplexity were not measured.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2609.03745) Sep 4, 2026

Significantly WORSE than the tuned baseline (+19.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 10.5129
Idea mean: 12.6084
Effect (Δ): 2.0955 (+19.9%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±9.2%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 0 configs

Limitations:

Only the registered tabular track was tested; vision/CIFAR, sequence, and dynamics were not tested. The analytic logarithmic utility slope was used instead of validation-loss perturbation estimates. Throughput, memory, kernel cost, and alternative code counts were not measured.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Failed ✗

Agent confidence: 9/10

The registered built-in tabular track was evaluated with mlp_tiny, eight paired seeds, a tuned baseline sweep, and parity-compliant idea learning rates. The trained-model mechanism signature was confirmed, but utility-weighted left-edge quantization significantly worsened test MSE, so the system did not work as a task-metric improvement.

Baseline: Friedman#1 tabular regression, mlp_tiny; baseline sweep lr=0.001 mean 32.339343, lr=0.003 mean 16.379407, lr=0.01 mean 10.050109; selected lr=0.01; full 8-seed mean MSE 10.512851.
Comparison: paired delta_mean=+2.095502, permutation p_value=0.0081, n_pairs=8, verdict='idea worse (significant)', system_worked=false
Idea: 16-code utility-weighted left-edge activation quantization; idea sweep means lr=0.003: 17.948323, lr=0.001: 49.849399, lr=0.01: 12.608353; selected lr=0.01; full 8-seed mean MSE 12.608353.

Benchmark result