Proximal Spherical Cubic Step

Integrate this as an optional inner solver for a selected parameter block, such as a final MLP layer, LoRA adapter, or small projection matrix; do not initially apply it to an entire large Transformer. At optimizer step k, flatten the block into x, compute g, Hessian-vector products, and approximate third-order directional contractions, then construct a low-rank tensor-contraction oracle for A rather than materializing a dense tensor. Choose a radius Delta and represent candidate directions with three unit vectors u,v,w using the augmented vector ilde{s}=(1,s). Initialize candidates from the negative-gradient direction, Adam's direction, and random unit vectors. For each PAM sweep, compute a_u=A(.,v,w) and update u <- normalize(beta_1 u-a_u); then compute a_v=A(u,.,w) and a_w=A(u,v,.) and apply the corresponding updates with beta_2 and beta_3. Decode the best candidate and scale it to ||s|| <= Delta. Evaluate rho=(f(x)-f(x+s))/(T_3(0)-T_3(s)); accept when rho >= eta, enlarge the radius for high rho, and shrink it after rejection. Estimate third derivatives using forward-over-reverse autodiff or finite differences of Hessian-vector products. First test a 2-layer MLP on CIFAR-10 against AdamW and a gradient-only cubic optimizer at matched gradient-equivalent FLOPs. Measure loss decrease per wall-clock second, final accuracy, gradient norm, rejected-step rate, and stability across seeds. Success is faster descent or fewer rejected steps on ill-conditioned blocks without loss spikes.