Эксперимент: KL-TopK Activation Bottleneck (#399)

{ "worked": true, "confidence": 8, "verdict": "Built a reproducible Gaussian KL-top-k activation bottleneck with calibration-time PCA, adaptive per-sample top-d retention, reconstruction, and numerical checks. The mechanism manifested: residual energy scaled linearly with covariance scale, isotropic inputs were rotation-invariant within Monte Carlo uncertainty, and PCA had lower residual than every tested random rotation across d=1..7. In the p=32 bottleneck, PCA retained 63.3%, 88.2%, and 97.8% of energy at d/p=0.094, 0.25, and 0.5, versus 39.8%, 70.6%, and 92.0% for a random rotation; the exact residual implementation error was 3.6e-15.", "metrics": { "baseline": "Random orthogonal rotation top-d: MSE 48.10, 23.53, and 6.42 at d/p=0.094, 0.25, and 0.5; retained-energy fractions 0.398, 0.706, and 0.920.", "idea": "Calibration PCA top-d: MSE 29.38, 9.42, and 1.78 at d/p=0.094, 0.25, and 0.5; retained-energy fractions 0.633, 0.882, and 0.978. Scale sweep residual/unit-scale ratios were 0.25, 0.5, 1, 2, and 4 as predicted." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 kl_topk_experiment.py", "files": [ "kl_topk_experiment.py", "results.json", "run_output.txt" ], "limitations": "This is a synthetic Gaussian reconstruction experiment, not an end-to-end neural-network task. Only random rotations were compared; no learned orthogonal rotation, non-Gaussian activations, covariance drift, refresh schedule, gradient estimator, metadata/index cost, wall-clock speed, or CUDA path was tested. Monte Carlo evidence cannot establish the paper's universal inequality, and the paper text indicates the exact constant-one conjecture remains unresolved." }