Log-Hölder Lyapunov Trust Region

(1) Integration point: use a gated RNN or state-space layer with $K$ discrete modes. The recurrent update is $h_{t+1}=A_{z_t}h_t+B_{z_t}u_t$, where $z_t$ follows a learned transition matrix $P$. Apply the controller to the recurrent matrices $A_i$ and transition logits, not to the input matrix.

(2) Pseudocode:

for each optimizer step:
    save A_old and P_old
    compute the ordinary gradient proposal A_new, P_new
    estimate l_plus_old, l_minus_old with QR products along mode paths
    estimate l_plus_new, l_minus_new for the proposal
    d = max_i spectral_norm(A_new[i]-A_old[i]) + max_abs(P_new-P_old)
    alpha = 0.5                 # general cocycle
    if conformal_mode: alpha = 1.0
    bound = C_hat / abs(log(max(d, d_min)))**alpha
    if abs(l_plus_new-l_plus_old) > bound or l_plus_new > stability_budget:
        backtrack the step, or multiply its learning rate by 0.5
    otherwise accept the proposal

Parameterize $P_{ij}=\exp(q_{ij})/\sum_{k\in N(i)}\exp(q_{ik})$ over allowed neighbors $N(i)$ so the transition graph remains fixed. Use QR reorthogonalization every few time steps; accumulate the logarithms of the diagonal of $R$ for the top exponent and use inverse-transpose products for the lower exponent.

(3) The logarithmic modulus, the exponent $\alpha=1/2$, and the conformal improvement $\alpha=1$ come from the paper. The constant $C$ is estimated empirically: collect checkpoint pairs, compute $d$ and observed $|\Delta\lambda_+|$, and choose a high quantile of $|\Delta\lambda_+||\log d|^\alpha$ as $\widehat C$. Define the zero-gap regime using $g=\max(\lambda_+-\lambda_-,0)<0.05$.

(4) First cheap experiment: train a two-mode width-64 RNN on permuted MNIST using AdamW, comparing the controller with ordinary AdamW at matched FLOPs across five seeds. Record exploding-gradient frequency, training loss versus steps, validation accuracy, estimated Lyapunov exponents, and accepted/backtracked update counts. The main falsifiable target is fewer divergent runs and the ability to use at least a 2x larger base learning rate without instability, with no loss in final validation accuracy.