# Эксперимент: Langevin Mean-Field Attention Heads (#28)

{
  "worked": false,
  "confidence": 9,
  "verdict": "Built an explicit particle-attention module with Euler–Maruyama parameter noise, residual attention updates, diversity diagnostics, particle dropping, and a reproducible synthetic classification benchmark. The diffusion variance matched 2βηT within 0.98% and the deterministic quadratic update matched its expected contraction. However, β=0 achieved 0.3346 validation loss and 95.31% accuracy, while β=0.001 and β=0.005 achieved losses 0.3644 and 0.4203 with accuracies 94.14% and 94.53%; diffusion also reduced half-particle accuracy, so no promised performance win was observed.",
  "metrics": {
    "baseline": "8 deterministic heads (β=0): validation loss 0.3346, accuracy 95.31%, half-head accuracy 94.14%, attention-map cosine 0.4793",
    "idea": "8 Langevin particles: β=0.001 loss 0.3644, accuracy 94.14%, half-particle accuracy 92.58%; β=0.005 loss 0.4203, accuracy 94.53%, half-particle accuracy 90.63%, attention-map cosine 0.3243"
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py",
  "files": [
    "experiment.py"
  ],
  "limitations": "Only a tiny synthetic classification task and 8 particles were tested; no WikiText-2, language modeling, large Transformer, matched-FLOP scaling study, annealing schedule, Adam effective-step analysis, or broader seed sweep was performed."
}