Эксперимент: Identity-Paired Progressive Depth (#421)

{ "worked": true, "confidence": 8, "verdict": "Built an additive-coupling identity-paired depth-growth module with an explicit inverse, plus reproducible math sweeps and a mini regression experiment. The mechanism manifested: inverse composition remained numerically identity, cancellation error increased with coupling scale, and untied output perturbation scaled linearly with step size. Insertion caused only a 7.0e-8 relative loss change, but the paired model did not beat continued shallow training after 300 updates in this tiny task.", "metrics": { "baseline": "Continued shallow model: final MSE 1.896e-14 after 300 post-expansion-equivalent updates.", "idea": "Identity-paired model: insertion relative loss change 7.018e-8; final MSE 5.052e-14 after 300 updates; best MSE 2.020e-14.", "random_deep": "Randomly initialized deeper model: final MSE 7.667e-5.", "cancellation_sweep": "Relative reconstruction error increased from 6.05e-9 at scale 0.01 to 6.52e-8 at scale 1.0.", "untied_scaling": "Output perturbation log-log slope 0.9995 across step sizes 1e-5 to 1e-3, matching the predicted slope of 1." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 identity_paired_experiment.py", "files": [ "identity_paired_experiment.py", "results.json" ], "limitations": "The experiment used a tiny synthetic regression problem rather than CIFAR-10, WikiText-2, or a Transformer. It tested one seed, one width, and one expansion, with no large-scale FLOP-normalized comparison or validation-based rollback sweep." }