Эксперимент: Coupled multilevel gradients for Markov-stream training (#556)

{ "worked": true, "confidence": 8, "verdict": "Built a reproducible coupled multilevel estimator toy verification with shared prefix blocks, inverse-probability corrections, and norm clipping, plus an ordered AR(1) streaming optimization benchmark. The mechanism manifested: telescoping passed within 2 standard errors, large-block mean-variance ratios matched the predicted (1+rho)/(1-rho) multiplier (observed 2.87/8.35/38.02 vs predicted 3/9/39), correction variance scaled approximately as 1/block, and clipping respected the bound exactly. Optimization showed lower final error and lower gradient lag-1 autocorrelation than the consecutive baseline at rho=0.8, but was worse at rho=0.95 and did not improve the iid case, so this is evidence for the mechanism rather than a uniform training win.", "metrics": { "baseline": "At rho=0.8: final parameter error 4.74e-5, lag-1 gradient autocorrelation 0.770; at rho=0.95: 4.87e-5 and 0.594.", "idea": "At rho=0.8: final parameter error 1.31e-5, lag-1 gradient autocorrelation 0.434, clipping 0.86%; at rho=0.95: 4.95e-4, autocorrelation 0.462, clipping 0.86%. Telescoping mean difference was 0.0062 with 2-SE tolerance 0.0322; clipping maximum norm was 1.0000000000000004 for bound 1." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json", "verify_extra.py" ], "limitations": "This is a two-parameter linear regression toy rather than a two-layer neural network or reinforcement-learning workload; AdamW, adaptive level-probability estimation, explicit mixing-time estimation, wall-clock profiling, and multiple random seeds were not tested. The benchmark uses fixed equal-size stream blocks and a simple SGD update, so it does not establish a general sample-efficiency or speed advantage." }