△ Mechanism confirmed, baseline not beaten 2026

Zero-Augmented Double-Scoring

Implementation & benchmark of arXiv:2607.20555 — Double-Scoring: Reliable Extraction of Strong Lottery Tickets

Usefulness7/10

Difficulty4/10

Novelty7/10

Architecture Accuracy Stability Scalability Initialization Training Pruning Linear algebra Optimization Combinatorics

Source paper: Double-Scoring: Reliable Extraction of Strong Lottery Tickets arXiv:2607.20555 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

For each frozen weight tensor, append a second tensor of identically shaped zero weights and assign trainable scores to both the real and dummy edges. Select a fixed number of candidates by top-k score in the doubled space; real edges selected by the competition remain active, while selected dummy edges consume the quota without changing the network. The resulting number of active original edges is learned rather than imposed by a separate layerwise sparsity search.

Formulas

$$f_W(x)=W_{\ell}\sigma_{\ell-1}\big(\cdots\sigma_1(W_1x+b_1)\cdots\big)+b_{\ell},$$

$$f_{W\odot H}(x),\qquad (W\odot H)_{ab}=W_{ab}H_{ab},\quad H_{ab}\in\{0,1\},$$

$$\widetilde W=[W\;0],\qquad \widetilde S=[S^{\mathrm{real}}\;S^{\mathrm{dummy}}],\qquad \widetilde H=\operatorname{TopK}_{K}(\widetilde S),$$

$$H_j=\\widetilde H_j\quad(1\\le j\\le M),\\qquad r=\\sum_{j=1}^{M}H_j,\\qquad \\rho_{\\mathrm{eff}}=\\frac{r}{M}\\leq\\frac{K}{M}.$$

Mathematical statement

The network is a composition of affine maps and activations, $$f_W(x)=W_{\\ell}\\sigma_{\\ell-1}\\big(\\cdots\\sigma_1(W_1x+b_1)\\cdots\\big)+b_{\\ell},$$ with frozen weights W_i. A binary mask H produces the subnetwork f_{W\\odot H}. The paper notes that masks are not unique when weights contain zeros: if W_{1,1}=0, changing the corresponding mask entry leaves W\\odot H unchanged, so distinct masks can represent the same masked tensor. Double-scoring exploits this redundancy by constructing an augmented candidate tensor \\widetilde W=[W\\;0] and score tensor \\widetilde S=[S^{\\mathrm{real}}\\;S^{\\mathrm{dummy}}]. For a layer containing M real edges, choose exactly K augmented coordinates using \\operatorname{TopK}_K(\\widetilde S). The realized original mask is H_j=1 exactly when selected coordinate j is a real edge; dummy selections contribute zero because their augmented weights are zero. Thus the effective real density is r/M, where r=\\sum_{j=1}^{M}H_j can vary between 0 and min(K,M), even though the augmented density K/(2M) is fixed. Any original mask with at most K active real edges can be represented by selecting those real coordinates plus K-r dummy coordinates. During score training, the forward pass is f_{\\widetilde W\\odot\\widetilde H} and only scores are updated; W remains fixed.

Implementation notes

Integrate this at every prunable linear or convolutional layer during score-only ticket extraction. Keep the usual randomly initialized weight tensor W frozen. For a layer with M scalar edges, create trainable real scores S_real with the same shape as W and trainable dummy scores S_dummy with the same shape; do not allocate or multiply a second dense weight tensor, because dummy values are mathematically zero. At each forward pass: (1) concatenate scores into S_aug = concat(S_real, S_dummy); (2) compute H_aug as a hard top-K indicator over all 2M scores, using K = floor(rho_aug times 2M); (3) form the effective real mask H = H_aug[..., :M]; and (4) execute the layer with W multiplied elementwise by H. Use a straight-through estimator, H_aug = stop_gradient(hard_topk(S_aug) - S_aug) + S_aug, or the same differentiable score surrogate used by the baseline edge-popup implementation. Update only S_real and S_dummy with SGD or Adam; after extraction, discard dummy coordinates and retain H. The paper supplies the zero-augmentation and fixed-density representation principle, so no additional mathematical quantity must be estimated. Log r/M independently for every layer and also log the fraction of selected candidates that are dummy. First run a cheap experiment on CIFAR-10 with a six-layer frozen random ReLU MLP or small CNN, comparing ordinary edge-popup at densities 0.1, 0.2, 0.4, and 0.6 against double-scoring at one fixed augmented density, using equal score-update steps and identical initialization. Evaluate test accuracy before weight training, accuracy after rewinding and training the selected topology, and variance across five seeds. Success means higher strong-ticket accuracy at comparable effective density, a flatter accuracy-versus-density curve, and substantially lower variance or sensitivity to the chosen density.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a readable zero-augmented double-scoring MVP with frozen weights, trainable real/dummy scores, hard Top-K straight-through selection, exhaustive representation checking, and a five-seed toy comparison. Across M=20,50,100 and augmented densities 0.2,0.5,0.8, observed real-count means and standard deviations matched the exact hypergeometric predictions, and dummy fractions were approximately 0.5; all masks with r≤K were represented exactly. The toy double-scoring accuracy was 0.700 versus 0.683 for baseline, but its effective density averaged 0.34 versus the baseline's fixed 0.30, so this is only a weak secondary signal rather than evidence of a clear accuracy win.

Agent confidence: 9/10

Baseline: Toy frozen linear score-only extraction: mean accuracy 0.6833, std 0.0581, effective density 0.300
Idea: Toy zero-augmented extraction: mean accuracy 0.7000, std 0.0586, effective density 0.340, effective-density std 0.0735
Mechanism sweep: 9 settings; absolute mean-count error <=0.0221 edges, standard-deviation error <=0.0245, observed dummy fraction 0.4991-0.5016 versus predicted 0.5
Representation: All 4096 masks for M=12 with r<=K=9 represented exactly; examples include r=0 with 9 dummy selections and r=9 with 0 dummy selections

Limitations: The planned CIFAR-10 six-layer MLP/CNN, rewinding and post-selection weight training, layerwise density curves, FLOP-matched comparisons, and five-seed trained-ticket evaluation were not tested. The toy comparison uses a single frozen linear layer and differs in effective density, so it cannot establish a practical strong-ticket improvement.

How to run: python3 run_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.20555) Sep 2, 2026

No significant difference vs. the tuned baseline (+0.4%, p=0.1671, wins 2 / 8 paired seeds).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 224.9028
Idea mean: 225.713
Effect (Δ): 0.8103 (+0.4%; negative = idea better)
Wins: 2 / 8 paired seeds
p-value: 0.1671 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 9 configs

Limitations:

Only the registered tabular track was tested. Strong-ticket rewinding, post-selection weight training, and layerwise density curves were not tested.

How to run: python3 run_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Zero-augmented double-scoring was implemented for frozen mlp_tiny weights on the registered tabular Friedman regression track. Baseline MSE was 224.903 and idea MSE was 225.713, giving delta_mean +0.810 and permutation p=0.16705, so the idea did not produce a significant win. The trained-model mechanism signature was confirmed: observed dummy fraction 0.50146 versus predicted 0.5 and observed effective real density 0.50350 versus predicted 0.5.

Baseline: MSE 224.90276336669922; best lr=0.01, density=0.75
Idea: MSE 225.713041305542; best lr=0.01, rho_aug=0.5
Paired: delta_mean=0.8102779388427734; idea_wins=2/8; p_value=0.16705

Benchmark result