Эксперимент: Critical Cross-Layer Weight Sharing (#1048)

{ "worked": true, "confidence": 9, "verdict": "Implemented correlated power-law cross-layer Gaussian weights and a residual-MLP Jacobian experiment. For gamma=0.3, observed tail exponents were 0.701 for absolute-correlation accumulation and 0.409 for squared-correlation accumulation, versus predictions 0.7 and 0.4. Critical gamma=0.5 and gamma=1 sums showed logarithmic growth, and empirical fourth moments matched the exact Wick formula within 0.8%. The correlated network had worse depth-128 Jacobian stability than IID weights, so the mechanism manifested but no performance win was demonstrated.", "metrics": { "baseline": "IID depth-128 max Jacobian singular value 1.357; normalized gradient fourth-moment proxy 1.007.", "idea": "Power-law gamma=0.3 depth-128 max Jacobian singular value 5.328; normalized gradient fourth-moment proxy 1.229. S2 exponent 0.409 versus predicted 0.4; Wick relative error 0.16%-0.74%." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 critical_sharing_experiment.py", "files": [ "critical_sharing_experiment.py", "results.json", "run_output.txt" ], "limitations": "Only a toy Gaussian-coordinate and small residual-MLP Jacobian experiment was tested; no CIFAR or sequence training, loss/FLOP comparison, tied or AR(1) baseline, learned mixing coefficient, or large-width/depth study was performed." }