# Эксперимент: Residual-Pivoted Kernel Attention (#1008)

{
  "worked": true,
  "confidence": 8,
  "verdict": "Built residual-diagonal sampled pivoted-Cholesky kernel attention with normalized low-rank output, plus uniform Nyström and random-column baselines. All three mechanism predictions were confirmed: expected trace-decrease errors were 1.27%, 0.43%, and 0.17%; diagonal-selection bias increased with spread; and an exact rank-8 PSD matrix reached 2.5e-15 relative residual after 8 pivots. In the synthetic attention test, residual pivoting had lower relative output error than both baselines at every tested rank, but this is not evidence of an end-to-end speed or memory win.",
  "metrics": {
    "baseline": "Uniform Nyström relative attention-output error: rank 8 55.7052, rank 16 19.4916, rank 32 6.7003. Random-column errors: 1.5512, 1.6235, 1.6475.",
    "idea": "Residual-pivot relative attention-output error: rank 8 1.0895, rank 16 0.6777, rank 32 0.3256. Mechanism checks: trace identity relative discrepancies 1.27%, 0.43%, 0.17%; diagonal-bias ratios to uniform 1.000, 1.085, 1.682, 3.070; rank-8 termination residual 2.54e-15."
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py",
  "files": [
    "experiment.py",
    "results.json",
    "REPORT.txt"
  ],
  "limitations": "Only a small synthetic N=128 attention experiment was tested. The kernel and full ground-truth matrix were materialized for evaluation; streaming kernel-column computation, GPU fallback, peak memory, wall-clock speedup, autoregressive decoding, transformer loss/perplexity, WikiText-2, and sequence lengths 512/2048 were not tested. The normalized low-rank factor can have signed entries, making denominators numerically fragile; no stabilization study beyond a small epsilon was performed."
}