Эксперимент: Invariant nonstandard residual blocks (#1401)

{ "worked": true, "confidence": 7, "verdict": "Built a denominator-damped multi-stage residual MLP and compared it with an explicit residual baseline on a fixed-seed digits task at h={0.5,1,2,4}. The scalar check clearly showed bounded nonstandard amplification while Euler became unstable for large h; at h=4, the idea reduced maximum activation norm (44.66 vs 100.96), gradient proxy (6.10 vs 43.56), and improved accuracy (0.9600 vs 0.9511), with no NaNs. This is a promising stability signal, but the block was about 3x slower and the experiment was small, so this is not evidence of a general efficiency or accuracy win.", "metrics": { "baseline": "At h=4: accuracy 0.9511, loss 0.2533, max activation 100.96, max gradient 43.56, runtime 6.29s; scalar Euler amplification reaches 3.0 at h=4 and 31.0 at h=32.", "idea": "At h=4: accuracy 0.9600, loss 0.1893, max activation 44.66, max gradient 6.10, runtime 17.88s; scalar nonstandard amplification is 0.333 at h=4 and 0.882 at h=32, remaining non-expansive in the tested range." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only the sklearn digits dataset and a small 16-block MLP were tested, with one random seed and 18 epochs. The implementation uses a simple two-stage explicit Heun-style construction rather than a fully implicit general-linear tableau; fixed-point residuals were logged but the stage update was not a true nonlinear implicit solve. FLOPs were not independently measured, and no CIFAR-10, neural ODE, sequence model, or multi-seed validation was performed." }