Smooth-RG Modewise Optimizer

1. Integration point: use this as a replacement for AdamW on a small MLP or Transformer, partitioning parameters into approximate curvature modes. Every 100 optimization steps, run eight Hutchinson/Lanczos probes to estimate the largest 64 Hessian eigenvalues $q_i$ or block-curvature modes. Measure each mode's effective relaxation response by applying a small perturbation $\delta\theta_i$ and recording the projected gradient change over 5–20 optimizer steps. 2. Pseudocode: initialize $s$ above the largest measured mode; at each update estimate $a_i=\langle\Delta g_i,\delta\theta_i\rangle/\langle\delta\theta_i,\delta\theta_i\rangle$, compute $R_s(q_i)=\alpha s^2/[\exp(q_i^2/s^2)-1]$, and update $\theta_i\leftarrow\theta_i-\eta g_i/(|a_i|+R_s(q_i)+\epsilon)$. Decrease $s$ geometrically only after the current spectral shell has been sampled, using no hard eigenvalue cutoff. 3. Computed from the paper: the smooth-regulator construction, scale-shell interpretation, and expected exponents. Estimated empirically: Hessian modes, damping $a_i$, and crossover locations. Fit $\log t_i$ against $\log q_i$ in sliding windows to estimate $z$. 4. First cheap experiment: train a width-256 MLP on MNIST and a 2-layer Transformer on a character-level copy task, comparing AdamW, Shampoo, and this optimizer at matched compute. Measure gradient explosions, loss decay, and mode relaxation times. The falsifiable signature is a slope change near shell transitions: $z\approx2$ at intermediate modes, $z\approx1$ where estimated $a_i$ approaches zero, and $z\approx1.5$ after low-mode coarse-graining. Each crossover should remain within 20% under changes in batch size or width.