Эксперимент: Saturation-Adaptive Prefill Chunking (#616)

{ "worked": true, "confidence": 8, "verdict": "Built a readable synthetic serving/power-trace experiment implementing the saturation/whale feedback controller, ramp quantile estimation, and largest-feasible-chunk selection. All four quantitative mechanism predictions passed: controller slope -28.0 versus -28 predicted, ramp reserve increased by 0.237 units/token, high-load reduction was 56.4% versus 8.6% at low load, and peak-power q99 changed only -3.0%. Against fixed chunking, adaptive chunking reduced q95 ramp by 45.8%, but also reduced the throughput proxy by 34.2%, so the benefit is a real ramp/throughput tradeoff rather than an unconditional serving win.", "metrics": { "baseline": "Fixed chunk=64: q95 ramp 28.73, peak q99 560.38, throughput proxy 64 tokens/quantum", "idea": "Adaptive mean chunk=42.14: q95 ramp 15.58 (-45.8%), peak q99 543.68 (-3.0%), throughput proxy 42.14 (-34.2%); largest tested chunk under ramp target 20 was 16" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py > results.json", "files": [ "experiment.py", "results.json" ], "limitations": "This is a controlled synthetic mechanism test, not an actual vLLM, TensorRT-LLM, or PyTorch LLM server benchmark. It does not measure real NVML power, GPU utilization, TTFT, end-to-end throughput, queueing effects, FLOPs, or model quality, and the peak-power invariance is enforced by the toy demand model rather than established on hardware." }