{
 "artifacts": null,
 "category": "stability",
 "description": "Use TT gauge freedom to enforce orthonormal interfaces during neural-network training, rather than allowing neighboring cores to develop arbitrarily large and small compensating scales. Periodically perform the paper's QR redistribution after gradient updates or local solves; this leaves the represented weight unchanged while improving the conditioning of subsequent core updates.",
 "formulas_latex": [
  "$$\\mathbf{G}_{k}=\\mathbf{Q}\\mathbf{R},\\qquad \\mathbf{Q}^{\\top}\\mathbf{Q}=\\mathbf{I},\\qquad \\mathbf{R}\\ \\text{upper triangular}$$",
  "$$\\mathcal{G}_{k}\\leftarrow\\operatorname{reshape}(\\mathbf{Q}),\\qquad \\mathcal{G}_{k+1}\\leftarrow\\mathbf{R}\\,\\mathcal{G}_{k+1}$$",
  "$$f(\\mathcal{G}_{k})=f(\\mathcal{G}_{k}^{\\star})-\\langle\\mathcal{R},\\mathcal{D}\\rangle+\\frac{1}{2}\\|\\mathcal{D}\\|_{\\mathrm{F}}^{2}$$",
  "$$\\mathbf{G}_{k,(12)}=\\mathbf{Q}\\mathbf{R},\\qquad \\mathbf{Q}^{\\top}\\mathbf{Q}=\\mathbf{I}_{r_k}$$"
 ],
 "id": 273,
 "implementation": "1. Integration point: apply gauge maintenance to every TT-matrix layer after each optimizer step, or every \\(K\\) steps for lower overhead. Operate on core parameter tensors, not activations; the transformation should be function-preserving up to floating-point error. For \\(G_k\\in\\mathbb{R}^{r_{k-1}\\times I_k\\times J_k\\times r_k}\\), flatten the first three axes into rows and the final rank axis into columns. 2. Pseudocode: after an AdamW or SGD update, for \\(k=1\\) to \\(n-1\\), set \\(M=\\operatorname{reshape}(G_k,(r_{k-1}I_kJ_k,r_k))\\); compute thin QR \\((Q,R)=\\operatorname{qr}(M)\\); set \\(G_k=\\operatorname{reshape}(Q,(r_{k-1},I_k,J_k,r_k))\\); reshape the next core so its first axis is \\(r_k\\), contract \\(R\\) with that axis, and restore its shape. Use positive-diagonal QR sign normalization to avoid needless sign flips. Optionally perform a reverse sweep with an RQ or LQ factorization to right-orthogonalize cores. 3. Compute QR directly from the paper's algebra. Estimate the benefit empirically using singular values of interface contractions, gradient norms per core, local design-matrix condition numbers, and the relative function difference \\(\\|W_{before}-W_{after}\\|_F/\\|W\\|_F\\). Do not clip \\(R\\) in the initial experiment because clipping changes the represented layer. 4. First cheap experiment: train TT-parameterized linear and two-layer MLP models on MNIST or CIFAR-10 using identical AdamW settings, comparing no gauge maintenance, QR every step, and QR every 10 steps. Track training loss, validation accuracy, per-core gradient variance, interface singular-value spread, optimizer stability, and function difference before versus after re-gauging. The expected signal is lower gradient imbalance and fewer NaNs or loss spikes at equal parameter count, while the function difference remains near floating-point tolerance.",
 "math_summary": "The paper updates a core by matricizing it as \\(\\mathbf{G}_k\\), computing the thin QR factorization \\(\\mathbf{G}_k=\\mathbf{Q}\\mathbf{R}\\), reshaping \\(\\mathbf{Q}\\) back into the current core, and absorbing \\(\\mathbf{R}\\) into the right-neighboring core. Here \\(\\mathbf{Q}\\) has orthonormal columns, \\(\\mathbf{Q}^{\\top}\\mathbf{Q}=\\mathbf{I}\\), and \\(\\mathbf{R}\\) is upper triangular. This is a gauge transformation: the product \\(\\mathbf{Q}\\mathbf{R}\\) equals the original matricized core, so the represented tensor or neural weight is unchanged in exact arithmetic. The resulting left-orthogonal gauge makes the contraction of cores up to position \\(k\\) numerically normalized. In the local residual expansion, \\(f(\\mathcal{G}_k)=f(\\mathcal{G}_k^\\star)-\\langle\\mathcal{R},\\mathcal{D}\\rangle+\\tfrac12\\|\\mathcal{D}\\|_F^2\\), where \\(\\mathcal{R}\\) is the residual at the local optimum and \\(\\mathcal{D}\\) is the output perturbation induced by \\(\\Delta\\mathcal{G}\\). Orthogonal interfaces reduce arbitrary scale factors in the local design operator, which should reduce per-core gradient imbalance and improve the conditioning of local least-squares systems.",
 "math_tags": [
  "tensor-decomposition",
  "linear-algebra",
  "numerical-analysis",
  "optimization"
 ],
 "ml_areas": [
  "mlp",
  "optimizer",
  "training-dynamics",
  "fine-tuning"
 ],
 "paper": {
  "arxiv_id": "2606.31061",
  "arxiv_url": "https://arxiv.org/abs/2606.31061",
  "summary_what_math_gives_to_ml": "The paper provides a constructive streaming update for tensor-train factors: solve each core exactly by least squares, then QR-orthogonalize it and absorb the triangular factor into the neighboring core. The transferable asset is the combination of gauge fixing, exact local minimization, and a residual identity showing that each local update is non-increasing in reconstruction error. This suggests replacing gradient updates for low-rank neural layers or adapters with streaming TT-ALS updates on minibatches of activation-target pairs, while using orthogonal cores to improve conditioning and prevent factor scale drift. The most credible first target is a frozen-feature linear or MLP projection, where the local least-squares problems are explicit and can be compared directly against LoRA and AdamW.",
  "title": "Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "accuracy",
  "memory"
 ],
 "title": "QR-gauged TT neural layer",
 "url": "https://synthcore.org/idea/273/qr-gauged-tt-neural-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}