Active Resolvent Regularization

1. Integration point: apply this to an RNN or linear state-space layer. For an RNN h[t+1] = tanh(W h[t] + U x[t] + b), estimate the local Jacobian J[t] = D[t]W, where D[t] = diag(1 − h[t+1]²). Construct a passive component H[t] = X[t]ᵀX[t]/n from a minibatch of hidden-state or Jacobian features, and define the measured residual K[t] = J[t] + H[t]. Initially use this decomposition only for monitoring and regularization rather than constraining the model exactly. 2. Pseudocode: estimate Jacobian-vector products; estimate the smallest eigenvalue δ_hat of H with Lanczos; estimate ||K||₂ by power iteration; solve (γI − J)v = r for several random Rademacher vectors r using conjugate gradients or a nonsymmetric linear solver; estimate χ_hat = mean(||v||²); add L_res = α[max(0, log(1 + χ_hat) − log(1 + χ_target))]². Alternatively reduce the recurrent learning rate when ε_hat/δ_hat = ||K||₂/δ_hat exceeds one. 3. Computed versus estimated: the Marchenko–Pastur gap is a theoretical reference valid only under approximately iid features; Jacobian norms, gaps, and resolvents must be estimated empirically. 4. First cheap experiment: train vanilla and resolvent-regularized GRUs on copy-memory and permuted-MNIST tasks with sequence lengths from 100 to 1000. Sweep an injected asymmetric perturbation W = Ws + εK. The falsifiable signature is a crossover near εc = δ_hat/||K||₂: below it, susceptibility should scale approximately as 1/δ_hat; above it, the growth should flatten or weaken. The measured crossover should be within 20 percent of this estimate, and regularization should reduce the 99th-percentile transient Jacobian gain without requiring a symmetric weight matrix.