Zero-mode MPS front-end for constrained sequence models

(1) Integration point: use an autoregressive Transformer for sequences obeying a known local rule, such as no adjacent 1s, finite-range exclusion, or legal symbolic actions. Add an MPS base distribution immediately before the final token softmax. Store one matrix $M_a\in\mathbb{R}^{D\times D}$ per token, start with bond dimension $D=2$, and maintain the prefix product $Q_t=M_{s_1}\cdots M_{s_{t-1}}$. For each candidate token $a$, compute its base score from the MPS contraction and set the score to $-\infty$ if appending $a$ violates the constraint. (2) Pseudocode: initialize $Q=I$; at position $t$, enumerate legal tokens; calculate $b_a=\log(|u^\top QM_av|^2+\epsilon)$; calculate Transformer residual $r_a=r_\theta(h_t,a)$; form $z_a=b_a+r_a$; normalize with log-sum-exp; train with teacher forcing or sample from the normalized distribution; then update $Q\leftarrow QM_{s_t}$. Use the supplied matrices as an initialization when token labels correspond to the local eigenbasis, and learn additional matrices for unmatched tokens. (3) Compute matrix products and legality masks exactly; optionally fine-tune $M_a,u,v$ with a small learning rate, while keeping the validity mask fixed. (4) First experiment: train a two-layer, four-head Transformer on a synthetic length-64 exclusion language and compare against the same model with a learned output head and against a model using only a hard validity mask. Match parameter count, optimizer, training steps, and processed tokens. (5) Pre-register mechanism tests: invalid generated sequences must remain below $10^{-6}$; at 10%, 25%, and 50% of baseline training tokens, the MPS model should achieve at least 5% lower validation NLL; and removing the MPS prior while retaining the hard mask should reduce loss improvement per token by at least 10% during the first 20% of training. Record NLL, invalid rate, gradient norm, and loss-versus-token curves. The transfer is falsified if the MPS prior gives no early-training improvement after parameter and FLOP matching, or if all gains disappear when the residual Transformer is sufficiently trained.