Эксперимент: Doubly-Stochastic Hyper-Residual Blocks (#885)

{ "worked": true, "confidence": 8, "verdict": "Built a self-contained Sinkhorn doubly-stochastic stream mixer and toy training comparison in experiment.py. The mechanism manifested: alternating normalization reduced row residual from 0.654 at K=1 to 0.00188 at K=20, arbitrary total stream mass error was 1.3e-15, and Frobenius mixing ratios stayed below 0.996 across logit scales. Training reduced stream RMS CV from 0.113 to 0.025 and eliminated gradient spikes above 10, but final loss was slightly worse than the unconstrained baseline (0.00681 vs 0.00629), so the stability/routing effect is real but no accuracy win was observed.", "metrics": { "baseline": "Final MSE 0.006289; best MSE 0.003276; 2 gradient spikes >10; stream RMS CV 0.1134; unconstrained row/column errors 1.50/1.50.", "idea": "Final MSE 0.006812; best MSE 0.003502; 0 gradient spikes >10; stream RMS CV 0.0250; row/column errors 0/0; arbitrary total-mass error 1.33e-15; maximum sampled Frobenius ratio 0.9955." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only a tiny synthetic regression task and one fixed seed were tested; no CIFAR-10, language modeling, deep transformer, equal-FLOP accounting, CUDA benchmark, or multi-seed statistical comparison was performed. Finite Sinkhorn iterations produce approximate rather than exact doubly-stochastic matrices; small spectral-norm excesses around 1.005 appeared from numerical residuals at extreme logits." }