Эксперимент: Rank-Safe Variable-Projection Gauss-Newton (#1369)

{ "worked": true, "confidence": 7, "verdict": "Built and numerically verified rank-safe truncated-pseudoinverse Gauss–Newton with exact variable projection for the linear output layer. The sanity check showed full inversion produced a step norm of 1e8 along a 1e-9 singular direction, while truncation kept the step norm at 0.1; regression test MSE was 0.00374 versus Adam's 0.27034. However, default hidden Gauss–Newton steps were all rejected, so the observed gain is primarily from exact output projection and rank safety rather than successful hidden-parameter optimization.", "metrics": { "baseline": "Adam: train loss 0.13666, test MSE 0.27034, runtime 2.42 s", "idea": "Rank-safe VP-GN: train loss 0.001881, test MSE 0.003736, runtime 9.45 s, 0 accepted hidden steps, maximum Jacobian condition number 2.82e18", "sanity": "Full pseudoinverse step norm 1.0e8 versus truncated step norm 0.1; truncated rank 1", "threshold_check": "rho=1e-5, 1e-4, and 1e-3 accepted 0/20 steps; rho=1e-2 accepted 20/20 with essentially unchanged test MSE 0.003754" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 rank_safe_vp_gn.py && /home/maxwelhelp/main/bin/python3 threshold_check.py", "files": [ "rank_safe_vp_gn.py", "threshold_check.py", "results.json", "threshold_results.json" ], "limitations": "Only a small 1D tanh regression problem was tested; no PDE operator, width-128 ReLU network, repeated seeds, wall-clock-matched optimization, or joint Gauss–Newton baseline was evaluated. The hidden Jacobian was formed densely, and no speedup was established; the accuracy improvement is confounded by exact least-squares output fitting." }