Secant-Calibrated lp Optimizer

Integrate this as a replacement optimizer for a small MLP or Transformer, operating independently on each parameter tensor or on flattened parameter groups. Start with p = 2 and then test p = 4, with q = p/(p-1). At initialization, choose a random normalized perturbation u for each parameter group and set z_0 = x_0 + delta u, where delta is a small relative perturbation. Compute gradients at x_0 and z_0. If their difference is below a numerical threshold tau, multiply delta by two and retry until M_0 is nonzero or a maximum perturbation is reached. Set M = M_0. At each iteration compute g, then d_i = -sign(g_i)|g_i|^(q-1)/||g||_q^(q-1), and trial parameters x_trial = x - alpha d/M. Compute the trial gradient g_trial. Accept if ||g_trial - g||_q <= c M ||x_trial - x||_p; otherwise reject, set M = 2M, and retry from the same parameters. After s consecutive accepted steps whose secant ratio is below c/4, set M = M/2. Use separate M values per layer, epsilon floors in all norms, and a maximum step size for numerical safety. The exact mathematical quantities are M_0, the dual-norm direction, and the secant acceptance test; L and R are not estimated. First compare against AdamW and normalized SGD on CIFAR-10 using a three-layer MLP, with equal numbers of backward passes. Record loss versus gradient evaluations, rejected-trial fraction, final gradient norm, wall-clock overhead, and sensitivity to a 100-fold change in the nominal alpha. A useful signal is faster loss decrease at equal gradient-query count, fewer loss spikes, and comparable or better validation accuracy. Then repeat with a small Transformer and vary target gradient tolerances to estimate empirical log-log complexity slopes.