Dual-unitary recurrent state block

Integrate the construction as the recurrent transition inside a linear state-space model or RNN, replacing the ordinary learned matrix A in x[t+1] = A x[t] + B z[t]. Split the hidden state into x = [u; v], with main-state dimension d and auxiliary dimension r. Parameterize U_plus as a real or complex unitary matrix using a product of Householder reflections, Givens rotations, or a Cayley transform. Construct D = block_diag(I_d, -I_r) and set U_minus = U_plus @ D at every forward pass. Use U_minus for the main recurrence: x[t+1] = U_minus @ x[t] + B @ z[t]. Optionally run a second recurrence with U_plus and combine the two outputs through a learned linear projection. Pseudocode: construct U_plus from trainable parameters; construct D; set U_minus = U_plus @ D; initialize x = 0; for each token z[t], set x = U_minus @ x + B @ z[t], then y[t] = C @ x. The unitary parameterization provides the exact norm-preservation property; dimensions, input scaling, output projection, and any coupling loss are empirical choices. First test on sequential MNIST and permuted sequential MNIST using a 256-state vanilla tanh RNN, a standard unitary RNN, and this dual-unitary block at matched parameter counts and FLOPs. Record hidden-state norms, gradient norms across sequence length, training loss after a fixed update budget, and validation accuracy when evaluating on sequences longer than those seen during training. Success means bounded hidden and gradient norms, no exponential degradation with context length, and equal-or-better long-context accuracy than the unconstrained RNN.