{
 "artifacts": null,
 "category": "architecture",
 "description": "Build a constrained autoregressive model whose initial logits are generated from a translation-invariant MPS associated with a local zero-mode construction. The MPS supplies a structured valid distribution before a Transformer residual is added, so the model starts on the constraint manifold instead of learning validity through a penalty.",
 "formulas_latex": [
  "$$H=\\mathcal{P}\\left(\\sum_j\\mathbf{h}_j\\right)\\mathcal{P}$$",
  "$$M^{L}_{1}=M^{R}_{1}=M^{U}_{1}=\\frac{1}{\\sqrt{6}}\\begin{pmatrix}0\u00260\\\\1\u00261\\end{pmatrix},\\qquad M_{1}^{O}=\\frac{\\pm 1}{\\sqrt{2}}\\begin{pmatrix}0\u00260\\\\1\u00261\\end{pmatrix}$$",
  "$$\\psi_M(s_1,\\ldots,s_n)=u^\\top M_{s_1}M_{s_2}\\cdots M_{s_n}v,\\qquad p_0(s)=\\frac{\\mathcal{P}(s)|\\psi_M(s)|^2}{\\sum_{s'}\\mathcal{P}(s')|\\psi_M(s')|^2}$$",
  "$$\\ell_\\theta(a\\mid s_{\u003ct})=\\log p_0(a\\mid s_{\u003ct})+r_\\theta(s_{\u003ct},a)-\\log\\sum_b p_0(b\\mid s_{\u003ct})e^{r_\\theta(s_{\u003ct},b)}$$"
 ],
 "id": 2959,
 "implementation": "(1) Integration point: use an autoregressive Transformer for sequences obeying a known local rule, such as no adjacent 1s, finite-range exclusion, or legal symbolic actions. Add an MPS base distribution immediately before the final token softmax. Store one matrix $M_a\\in\\mathbb{R}^{D\\times D}$ per token, start with bond dimension $D=2$, and maintain the prefix product $Q_t=M_{s_1}\\cdots M_{s_{t-1}}$. For each candidate token $a$, compute its base score from the MPS contraction and set the score to $-\\infty$ if appending $a$ violates the constraint. (2) Pseudocode: initialize $Q=I$; at position $t$, enumerate legal tokens; calculate $b_a=\\log(|u^\\top QM_av|^2+\\epsilon)$; calculate Transformer residual $r_a=r_\\theta(h_t,a)$; form $z_a=b_a+r_a$; normalize with log-sum-exp; train with teacher forcing or sample from the normalized distribution; then update $Q\\leftarrow QM_{s_t}$. Use the supplied matrices as an initialization when token labels correspond to the local eigenbasis, and learn additional matrices for unmatched tokens. (3) Compute matrix products and legality masks exactly; optionally fine-tune $M_a,u,v$ with a small learning rate, while keeping the validity mask fixed. (4) First experiment: train a two-layer, four-head Transformer on a synthetic length-64 exclusion language and compare against the same model with a learned output head and against a model using only a hard validity mask. Match parameter count, optimizer, training steps, and processed tokens. (5) Pre-register mechanism tests: invalid generated sequences must remain below $10^{-6}$; at 10%, 25%, and 50% of baseline training tokens, the MPS model should achieve at least 5% lower validation NLL; and removing the MPS prior while retaining the hard mask should reduce loss improvement per token by at least 10% during the first 20% of training. Record NLL, invalid rate, gradient norm, and loss-versus-token curves. The transfer is falsified if the MPS prior gives no early-training improvement after parameter and FLOP matching, or if all gains disappear when the residual Transformer is sufficiently trained.",
 "math_summary": "The paper represents the constrained Hamiltonian as $H=\\mathcal{P}(\\sum_j\\mathbf{h}_j)\\mathcal{P}$, where $\\mathbf{h}$ is the single-site unconstrained Hamiltonian, $j$ indexes sites, and $\\mathcal{P}$ projects onto globally allowed configurations. Its construction selects local MPS tensors from eigenspaces of the local operator. The extracted example contains $M^{L}_{1}=M^{R}_{1}=M^{U}_{1}=\\frac{1}{\\sqrt{6}}\\begin{pmatrix}0\u00260\\\\1\u00261\\end{pmatrix}$ and $M_{1}^{O}=\\frac{\\pm1}{\\sqrt{2}}\\begin{pmatrix}0\u00260\\\\1\u00261\\end{pmatrix}$. For a sequence $s=(s_1,\\ldots,s_n)$, let $M_a\\in\\mathbb{R}^{D\\times D}$ be the tensor for token $a$, and define the open-boundary MPS amplitude $\\psi_M(s)=u^\\top M_{s_1}\\cdots M_{s_n}v$, where $u,v\\in\\mathbb{R}^{D}$ are boundary vectors. The adapted base distribution is $p_0(s)=\\mathcal{P}(s)|\\psi_M(s)|^2/Z$, with $Z=\\sum_{s'}\\mathcal{P}(s')|\\psi_M(s')|^2$. The Transformer learns a residual score $r_\\theta(s_{\u003ct},a)$ on top of the MPS conditional distribution. The mechanism is exact support restriction through $\\mathcal{P}$ plus a low-bond-dimension tensor prior whose amplitudes are computed by matrix products.",
 "math_tags": [
  "linear-algebra",
  "tensor-decomposition",
  "algebra",
  "dynamical-systems"
 ],
 "ml_areas": [
  "transformer",
  "embedding",
  "data-augmentation"
 ],
 "paper": {
  "arxiv_id": "2608.31165",
  "arxiv_url": "https://arxiv.org/abs/2608.31165",
  "summary_what_math_gives_to_ml": "The paper gives a constructive method for generating translation-invariant matrix-product states that are exact zero modes of constrained Hamiltonians. Its transferable asset is the combination of a hard global validity projector with low-bond-dimension local tensors selected from the eigendecomposition of an unconstrained local operator. In neural sequence models, this can become a structured base distribution or finite-state front-end that guarantees valid outputs and supplies an algebraic prior before a Transformer learns task-specific residuals. The strongest initial test is on synthetic exclusion languages, where validity, early optimization, and sample-efficiency predictions can be measured exactly.",
  "title": "Efficient search for excitable zero-modes in constrained systems",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 6,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "sample-efficiency",
  "accuracy"
 ],
 "title": "Zero-mode MPS front-end for constrained sequence models",
 "url": "https://synthcore.org/idea/2959/zero-mode-mps-front-end-for-constrained-sequence-models",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}