Эксперимент: Fused truncated-power KAN activation (#1209)

{ "worked": false, "confidence": 9, "verdict": "Built a PyTorch fused truncated-power cubic spline evaluator with bounded coordinate clamping, Cox–de Boor reference evaluation, autograd derivative checks, precision diagnostics, and CUDA timing. The formulas agree to approximately 1.7e-12 max error in float64, but float32 cancellation grows to 9.1e-4 output error and 1.17e-3 gradient error near the upper bounded coordinate range. The proposed evaluator was only 1.30x faster in forward and 1.26x faster for forward-plus-backward, so the claimed substantial speedup and float32 accuracy target were not observed.", "metrics": { "baseline": "Cox–de Boor: 2.623 ms forward, 5.538 ms forward+backward; float32 comparison error 9.07e-4 max output and 1.17e-3 max gradient.", "idea": "Truncated-power: 2.020 ms forward, 4.397 ms forward+backward; 1.30x and 1.26x speedups respectively. Float64 max error 1.70e-12, but float32 equivalence is insufficiently accurate." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "No end-to-end KAN training or validation-accuracy experiment was run. The implementation is vectorized PyTorch rather than a custom Triton/CUDA fused kernel or torch.compile graph, so these timings do not establish the performance of a true kernel fusion. The float32 cancellation issue was measured but not mitigated with compensated arithmetic or a numerically stable alternative." }