Energy-Adaptive Inertial Optimizer

Integrate this as a PyTorch optimizer for an MLP, CNN, transformer, or parameter-efficient fine-tuning problem. Maintain a velocity tensor for every trainable parameter. Begin with K = 0, giving S_k = ||v_k||^2 + epsilon; only add layer-graph coupling after the scalar mechanism is validated. Let V equal the minibatch loss, optionally plus lambda times the squared distance from the initial parameters. At each step, compute the minibatch gradient g_k, flatten or aggregate all velocity tensors, calculate S_k = sum_i v_k,i^2 + epsilon, set c_k = gamma times S_k^q, and update v_k using v = v - eta*g - eta*c_k*v. Then update theta using theta = theta + eta*v. Also test the semi-implicit damping update v = (v - eta*g)/(1 + eta*c_k), which is safer when damping becomes large. Quantities computed directly are the gradient, velocity norm, damping coefficient, and empirical energy. The paper gives the continuous Lyapunov law, not an unconditional discrete-time guarantee. First train a two-layer MLP on MNIST with SGD-momentum, fixed-friction heavy-ball, and the adaptive optimizer, matching learning rates and parameter counts. Log E_k and D_k = gamma*S_k^q*||v_k||^2. For small eta, predict E_(k+1) - E_k approximately equal to -eta*D_k plus an order-eta-squared error; regression of energy change against -eta*D_k should have slope near one. Sweep eta and record the first point where fewer than 95 percent of steps decrease energy. The adaptive method should shift the oscillation or divergence boundary upward, while excessively large gamma should visibly reduce progress near minima.