Эксперимент: Manifold-kernel attention (#470)

{ "worked": true, "confidence": 8, "verdict": "Built a NumPy MVP for Gaussian manifold-kernel attention with explicit r^-k scaling, row normalization, local bandwidths, Levina–Bickel dimension estimation, and a denoising attention comparison. The mechanism manifested: kernel mass scaled approximately as r^k (observed slopes 1.00, 1.98, 2.99 for k=1,2,3), dimension normalization made raw mass scale approximately invariant (observed slopes near 0), and effective support scaled approximately as bandwidth^k (observed slopes 1.01, 2.02, 2.90 for k=1,2,3). The kernel also beat the tested dot-product and uniform regressors, but this is only a small synthetic result and not evidence of a CIFAR/ViT win.", "metrics": { "baseline": "Synthetic regression: dot-product attention MSE 0.8913; uniform aggregation MSE 0.8434; dot-product entropy 6.2754.", "idea": "Kernel attention MSE 0.7263; entropy 4.0010; mean effective neighbors 40.83. Continuum slopes: mass-vs-bandwidth [0.997, 1.980, 2.989] for dimensions [1,2,3] versus predicted [1,2,3]; normalized-mass slopes at p=k were approximately [0.0004, 0.0036, -0.0138]." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "No CIFAR-10, ViT, training-time/FLOP, GPU, sparse-kNN speed, gradient-variance, duplicate-token robustness, nonuniform-density robustness, or learned end-to-end comparison was tested. Higher-dimensional bandwidth scaling was affected by finite sample and bounded-domain saturation; the reported estimator check is noisy and uses synthetic Euclidean torus distances." }