# Эксперимент: Exact doubly stochastic low-rank attention (#814)

{
  "worked": true,
  "confidence": 8,
  "verdict": "Built an exact shared-marginal KL projection for positive low-rank factors, with gauge-fixed Newton updates, analytic covariance Hessian, matrix-free HVP, and thin value application. Row and shared-column residuals stayed near 1e-14–1e-13 across n=16–256 and ranks 2–16; HVP relative error was about 2e-16–1e-13 with positive reduced curvature. Thin application matched explicitly formed factorized W to about 3e-16–7e-16, while factor storage was 3.94–15.98x smaller and application was 5.16x faster at n=512,r=16. This does not establish unchanged model quality because no Transformer training comparison was run.",
  "metrics": {
    "baseline": "Explicit factorized W application: 1.282 ms median and 262144 attention-state entries at n=512,r=16; no end-to-end standard softmax training benchmark.",
    "idea": "Thin U((V^T X)/g) application: 0.249 ms median, 5.16x faster; 16400 factor entries, 15.98x lower state storage; output relative error versus explicit factorized W 6.42e-16; doubly-stochastic residual 7.46e-14."
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 verify.py",
  "files": [
    "exact_ds.py",
    "verify.py",
    "results.json"
  ],
  "limitations": "Only toy numerical verification and application benchmarks were tested. No end-to-end Transformer, WikiText/CIFAR validation loss, equal-parameter comparison, optimization stability study, GPU benchmark, or comparison to standard softmax attention quality was performed; projection cost was excluded from timing."
}