Эксперимент: Ranking-Aware Projection Dimension Rule (#1224)

{ "worked": false, "confidence": 9, "verdict": "Built a reproducible ranking-aware projection module with exact Beta–arcsine evaluation, Monte Carlo verification, adaptive dimension expansion, and a retrieval benchmark. The core mathematical phenomenon was clearly observed: empirical Kendall correlation matched the exact law and scaled approximately as (2/pi)sqrt(m/d). However, on the toy retrieval task the ranking-rule projection used 80/128 dimensions, reducing memory by 37.5%, but reduced Recall@10 from 1.00 to 0.631 and Recall@1 to 0.303, so the proposed compression did not deliver the promised fixed-quality win.", "metrics": { "baseline": "JL-style sizing: m=128/128, memory ratio=1.000, Recall@1=1.000, Recall@10=1.000, top-20 Kendall=1.000, query time=0.746 s", "idea": "Ranking rule: m=80/128, memory ratio=0.625, Recall@1=0.303, Recall@10=0.631, top-20 Kendall=0.410, query time=0.562 s; calibration tau=0.574 at m=80" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json", "run_output.txt" ], "limitations": "This was a small synthetic clustered retrieval benchmark with d=128, 4,000 indexed items, and 300 queries, not the proposed 100k-item d=768 benchmark or FAISS implementation. Only random Gaussian projections and squared-Euclidean retrieval were tested; no learned projections, real embeddings, top-q asymptotic agreement study, confidence intervals, GPU execution, or large-scale latency/memory profiling was evaluated." }