Critical-Orbit Trust Region for Chebyshev Optimizers

Integrate this as an optional inner update for a neural optimizer rather than replacing minibatch SGD everywhere. At parameter vector theta, choose a normalized direction u=-P g_hat/||P g_hat||, where g_hat is a minibatch gradient and P is a diagonal Adam-style preconditioner. Probe the directional stationarity function p(t)=u^T grad L(theta+t u) at five to nine small values of t, using the same minibatch or common random numbers. Fit a degree-3 or degree-5 polynomial p_hat(t), differentiate it analytically, and apply the Chebyshev update. Find the real roots of the derivative C_hat'(t) with a polynomial root solver; discard poles, points outside the trust interval [-r,r], and critical points close to the selected root t_*. Compute Gamma from the fitted polynomial. If Gamma exceeds 0.9, halve r and set alpha to 0.5 alpha. If Gamma is below 0.5, increase r by 10 percent up to a cap. Reject any update that increases a fresh validation minibatch loss. The exact Chebyshev map and the contraction-expansion mechanism come from the paper; polynomial fitting, root selection, and the threshold are empirical adaptations. First test on a two-layer MLP fitting a one-dimensional polynomial regression dataset, comparing SGD, Adam, and undamped Chebyshev updates. Sweep trust radii and record divergence. The quantitative prediction is a transition near Gamma=1: Gamma<1 should produce bounded directional iterates and cubic local stationarity convergence, whereas Gamma>1 should produce more basin exits or exploding steps. On the exact synthetic family p_n, the implementation should recover the transition between n=16 and n=17 when D_n crosses one.