Projective Jacobian Compensation

Integrate this into the optimizer update for a small parameter block, such as each Transformer MLP projection or each residual branch, rather than into the forward activations. Maintain a cheap reference perturbation delta in the same block, normalized after every update, and use k controllable directions U=[u_1,...,u_k]. These directions can be fixed random Gaussian vectors, gradient-history vectors, or orthogonal four-dimensional rotations generated by the matrices M_j from the paper. At every T training steps: (1) estimate J delta with one extra Jacobian-vector product, implemented with forward-mode autodiff or finite differences J delta approximately equal to (Phi(theta+eps delta)-Phi(theta))/eps; (2) solve the k-by-k ridge system a=-(U^T U+lambda I)^(-1)U^T J delta; (3) apply the controlled optimizer update theta_new=theta_optimizer - eta U a, clipping ||a|| and optionally multiplying it by a blockwise trust coefficient; (4) update delta using the same local Jacobian, normalize delta, and refresh U every R steps. The low-rank correction is the neural analogue of the paper's finite-rank Wiener-path perturbation. Estimate the mathematical quantities empirically: J delta comes from autodiff, the residual contraction statistic is log(||J delta+U a||/||delta||), and the control cost is ||a||^2. First test on CIFAR-10 with a 6-layer MLP or small ViT against AdamW at matched forward FLOPs, using k=4,8, lambda in {1e-4,1e-3,1e-2}, and T in {10,50}. Measure loss descent, gradient-noise sensitivity, validation accuracy, and the empirical mean log-residual over 1,000 blocks. Success means lower residual growth and fewer loss spikes at equal final accuracy; a negative residual statistic together with faster early loss descent is the primary falsifiable signal. Also run an ablation with U absent, isotropic Gaussian noise, and ordinary gradient clipping.