Эксперимент: Late-Time Fractional-Order Optimizer (#1272)

{ "worked": false, "confidence": 9, "verdict": "Built a late-time power-law order estimator and a piecewise-constant fractional-memory optimizer in PyTorch, with CUDA fallback and fixed-order controls. The synthetic math check recovered alpha=0.68 as 0.690 (absolute error 0.010), confirming the ratio formula. However, on the small quadratic task adaptive fractional memory reached tail loss 2.711 versus AdamW 0.00977, so the proposed optimizer showed no win and was substantially less stable in this test.", "metrics": { "baseline": "AdamW: final loss 0.00806, tail-20 loss 0.00977, loss at step 40 0.260; runtime 0.56 s.", "idea": "Adaptive fractional memory: final loss 1.721, tail-20 loss 2.711, loss at step 40 0.499; estimated alpha ended at 0.901 (range 0.5-0.901), runtime 1.32 s. Math check: true alpha 0.68, estimate 0.690, error 0.010." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 run_experiment.py", "files": [ "fractional_optimizer.py", "run_experiment.py" ], "limitations": "Only a small full-batch quadratic regression was tested, not CIFAR-10, a neural network, minibatch noise, multiple random seeds, FLOP-matched training, gradient variance, or sensitivity studies. The fractional update uses normalized finite gradient history and does not implement AdamW diagonal normalization, so this is an MVP rather than a faithful production AdamW replacement." }