Эксперимент: Tail-Weighted Optimal Batch Scheduling (#587)

{ "worked": true, "confidence": 8, "verdict": "Built a NumPy MVP implementing the theorem’s tail coefficients, continuous optimum, bounded integer allocation, and a noisy-quadratic SGD comparison. The mechanism was numerically confirmed: allocation ratios matched sqrt(c_t s_t) to 2.2e-16, the continuous solution matched the predicted optimum exactly, and scaling noise by 0.25/1/4/16 scaled the minimized objective by exactly the same factors while leaving relative allocations unchanged. In the toy training run, tail weighting improved step-32 loss and the theoretical batch-dependent bound versus static batching, but had slightly worse final stochastic loss (0.00883 vs 0.00873), so this is mechanism-confirmed rather than a demonstrated end-to-end win.", "metrics": { "baseline": "Static batch: final loss 0.0087314 ± 0.0006217 SE; step-32 loss 0.0292732; theoretical batch objective 0.0364224.", "idea": "Tail-weighted integer batch: final loss 0.0088268 ± 0.0006200 SE; step-32 loss 0.0283912; theoretical batch objective 0.0326175 (10.44% lower than static); integer/continuous objective ratio 1.00043. Allocation was batch 26 at step 1 and 18 at step 64, with range 8–26." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 tail_batch_experiment.py", "files": [ "tail_batch_experiment.py", "results.json", "run_output.txt" ], "limitations": "This was a small synthetic noisy strongly-convex quadratic, not a neural network or real data-loader experiment. Noise scales were prescribed rather than estimated from probe microbatches; wall-clock overhead, GPU memory, gradient accumulation, AdamW, validation loss, and real sample-efficiency were not tested. The tail-allocation identity is mathematically exact under the assumed second-moment model, while practical gains may depend on estimation error and integer/clipping constraints." }