Hartogs Core Regularizer for Two-Axis State Transitions

(1) Integration point: add the regularizer to a small linear state-space or two-dimensional recurrent model. Use d-dimensional square matrices A and B as the two state transitions. For a 1D sequence use h_{t+1}=tanh(Ah_t+Bz_t+Cu_t); for a grid or raster task use h_{i,j}=tanh(Ah_{i-1,j}+Bh_{i,j-1}+Cu_{i,j}). Apply the penalties only to A and B, not to input or output projections. (2) Pseudocode: initialize A and B near scaled orthogonal matrices. On each minibatch compute H = B@B.T - A@A.T - B@B@B.T@B.T + A@B@A.T@B.T, set Hs=(H+H.T)/2, and compute the eigenvalues of Hs. For d<=128 use a full symmetric eigendecomposition; for larger d use Lanczos to estimate the k smallest eigenvalues. Set negative_eig_loss = sum(relu(-eigenvalues)**2), iso_loss = ||A.T@A-I||_F^2 + ||B.T@B-I||_F^2, and comm_loss = ||A@B-B@A||_F^2. Optimize task_loss + lambda_iso*iso_loss + lambda_com*comm_loss + lambda_core*negative_eig_loss with AdamW. (3) Computed from the paper: the four-term core operator and its positive-semidefinite target. Estimated empirically: finite-matrix eigenvalues, regularization weights, and whether exact isometry is beneficial. Ramp lambda_core from zero during the first 10 percent of training and clip gradients. (4) First experiment: train d=64 models on permuted-MNIST, adding-memory, and copy tasks, comparing vanilla RNN, orthogonal RNN with L_iso, L_iso+L_com, and the full core regularizer at matched parameter count and FLOPs. Measure validation accuracy, gradient norms across sequence length, spectral-radius drift, minimum eigenvalue of Hs, and loss after 1,000-step extrapolation. The hypothesis is reduced exploding or vanishing gradients and better long-horizon accuracy than commutation-only control; reject the method if it gives no gain at equal tuning cost.