Bartlett-LKJ Correlated Head Noise

Integrate this at the perturbation-generation point rather than inside the core attention matrix multiplication. For a transformer with H heads, set n=H and generate one lower-triangular Bartlett factor B per training step, microbatch, or epoch. Use eta as a fixed hyperparameter or a schedule. Pseudocode: input head outputs h[1:H], noise scale sigma, and eta; initialize A=0; for i=2,...,H, sample A[i,j]~Normal(0,1) for j<i and A[i,i]~sqrt(ChiSquare(2*eta+i-2)); set B[i]=A[i]/norm(A[i]); set B[1,1]=1; sample z~Normal(0,I_H); set eps=sigma*(B@z); replace independent head noise eps_i with eps_i, or use eps_i to scale head dropout logits. For multiple feature dimensions, draw z independently per token and feature group, reusing B across the sequence to create head-level rather than token-level correlation. The exact paper-derived computation is the Bartlett degrees-of-freedom rule and row normalization; no empirical eigendecomposition is needed. Record head-output correlation and gradient variance empirically. First experiment: compare a 6-head, 2-layer Transformer on CIFAR-10 or WikiText-2 using independent dropout versus LKJ-correlated dropout at eta in {0.3,1,3,10}, matched noise variance and FLOPs. Measure validation accuracy or perplexity, gradient variance, and robustness across seeds. Success means improved accuracy or calibration at equal cost, or lower seed-to-seed variance without optimization instability.