{
 "artifacts": null,
 "category": "optimization",
 "description": "Parameterize a frozen-feature neural projection as a tensor-train operator and update one TT core at a time by an exact least-squares solve on each incoming minibatch. After every core solve, QR-orthogonalize its matricization and absorb the triangular factor into the next core, preserving the represented operator while controlling conditioning. This creates a deterministic, low-memory alternative to Adam-trained LoRA for regression heads, MLP projections, or linear attention projections.",
 "formulas_latex": [
  "$$\\mathcal{X}[t]=\\mathcal{X}[t-1]\\boxplus\\overline{\\mathcal{X}}_{t}$$",
  "$$f(\\mathcal{G}_{k})=\\frac{1}{2}\\left\\|\\widehat{\\mathcal{X}}_{t}-\\mathbf{A}_{k}[t]\\times_{k}^{1}\\mathcal{G}_{k}\\times_{k+1}^{1}\\mathbf{B}_{k}[t-1]\\right\\|_{\\mathrm{F}}^{2}$$",
  "$$\\mathcal{G}_{1}[t]=\\widehat{\\mathcal{X}}_{t}\\times\\mathbf{B}_{1}[t-1]^{\\top},\\qquad \\mathbf{G}_{1}[t]=\\mathbf{Q}\\mathbf{R},\\qquad \\mathbf{Q}^{\\top}\\mathbf{Q}=\\mathbf{I}$$",
  "$$f(\\mathcal{G}_{k})=f(\\mathcal{G}_{k}^{\\star})-\\sum_{l,m,n}\\mathcal{R}(l,m,n)\\mathcal{D}(l,m,n)+\\frac{1}{2}\\|\\mathcal{D}\\|_{\\mathrm{F}}^{2}$$"
 ],
 "id": 272,
 "implementation": "1. Integration point: replace a frozen-feature linear layer or MLP projection \\(y=Wx\\) by a TT-matrix operator. Factor input and output dimensions into mode sizes \\(I_1\\cdots I_n\\) and \\(J_1\\cdots J_n\\), and represent \\(W\\) with cores \\(G_k\\in\\mathbb{R}^{r_{k-1}\\times I_k\\times J_k\\times r_k}\\), where \\(r_0=r_n=1\\). Keep the backbone frozen and train only the TT cores on minibatches \\((x_b,y_b)\\). 2. Pseudocode: initialize TT cores and cache right contractions \\(B_k\\); for each minibatch, contract all cores except \\(G_k\\) to form design operator \\(Z_k\\), solve \\(\\operatorname{vec}(G_k^\\star)=\\arg\\min_g\\|Z_kg-y\\|_2^2+\\lambda\\|g\\|_2^2\\) using Cholesky or conjugate gradients; reshape \\(G_k^\\star\\); matricize it; compute thin QR \\((Q,R)=\\operatorname{qr}(G_{k,(12)})\\); replace the core by reshaped \\(Q\\), and multiply \\(R\\) into the next core along its left-rank axis. Sweep \\(k=1,\\ldots,n\\), then optionally sweep backward. 3. The paper supplies the local least-squares structure, QR gauge transformation, and monotone local-objective argument. Estimate \\(\\lambda\\), minibatch covariance, and rank from validation data; use a rolling buffer or exponential weighting if the stream is nonstationary. Monitor residual before and after each update and reject a solve if approximate contractions increase the residual. 4. First experiment: use CIFAR-10 or ImageNet-100 with a frozen ViT or MLP feature extractor and replace its classifier or one projection layer. Compare TT-ALS against equal-parameter LoRA trained by AdamW and full-layer SGD. Begin with rank 4--8 and 4--6 modes. Measure wall-clock time, peak memory, validation accuracy, and loss after equal numbers of examples. The expected signal is faster early loss descent and lower optimizer memory, with equal accuracy at less than half the update time.",
 "math_summary": "The paper represents a streaming tensor as \\(\\mathcal{X}[t]=\\mathcal{X}[t-1]\\boxplus\\overline{\\mathcal{X}}_t\\), where \\(\\overline{\\mathcal{X}}_t\\) is the new tensor slice. A TT operator consists of cores \\(\\mathcal{G}_k\\), with left interface \\(\\mathbf{A}_k[t]\\) and right interface \\(\\mathbf{B}_k[t-1]\\); with all cores except \\(\\mathcal{G}_k\\) fixed, the local objective is \\(f(\\mathcal{G}_k)=\\tfrac12\\|\\widehat{\\mathcal{X}}_t-\\mathbf{A}_k[t]\\times_k^1\\mathcal{G}_k\\times_{k+1}^1\\mathbf{B}_k[t-1]\\|_F^2\\). Here \\(\\widehat{\\mathcal{X}}_t\\) is the current minibatch target tensor, \\(\\mathbf{A}_k[t]\\) contracts all cores to the left of core \\(k\\), and \\(\\mathbf{B}_k[t-1]\\) contracts all cores to its right. The updated core \\(\\mathcal{G}_k^\\star\\) satisfies \\(\\partial f/\\partial\\mathcal{G}_k=0\\). For any perturbation \\(\\mathcal{G}_k=\\mathcal{G}_k^\\star+\\Delta\\mathcal{G}\\), define \\(\\mathcal{R}=\\widehat{\\mathcal{X}}_t-\\mathbf{A}_k[t]\\times_k^1\\mathcal{G}_k^\\star\\times_{k+1}^1\\mathbf{B}_k[t-1]\\) and \\(\\mathcal{D}=\\mathbf{A}_k[t]\\times_k^1\\Delta\\mathcal{G}\\times_{k+1}^1\\mathbf{B}_k[t-1]\\). The paper obtains \\(f(\\mathcal{G}_k)=f(\\mathcal{G}_k^\\star)-\\langle\\mathcal{R},\\mathcal{D}\\rangle+\\tfrac12\\|\\mathcal{D}\\|_F^2\\); stationarity makes the cross term zero, proving local objective decrease. For the first core, the exact update simplifies to \\(\\mathcal{G}_1[t]=\\widehat{\\mathcal{X}}_t\\times\\mathbf{B}_1[t-1]^\\top\\). Its matricization is factorized as \\(\\mathbf{G}_1[t]=\\mathbf{Q}\\mathbf{R}\\), with \\(\\mathbf{Q}^\\top\\mathbf{Q}=\\mathbf{I}\\); reshape \\(\\mathbf{Q}\\) into the core and absorb \\(\\mathbf{R}\\) into the next core. The neural adaptation solves the same least-squares problem for a tensorized weight operator mapping feature tensor \\(x\\) to target \\(y\\).",
 "math_tags": [
  "tensor-decomposition",
  "linear-algebra",
  "optimization",
  "numerical-analysis"
 ],
 "ml_areas": [
  "mlp",
  "fine-tuning",
  "optimizer",
  "compression"
 ],
 "paper": {
  "arxiv_id": "2606.31061",
  "arxiv_url": "https://arxiv.org/abs/2606.31061",
  "summary_what_math_gives_to_ml": "The paper provides a constructive streaming update for tensor-train factors: solve each core exactly by least squares, then QR-orthogonalize it and absorb the triangular factor into the neighboring core. The transferable asset is the combination of gauge fixing, exact local minimization, and a residual identity showing that each local update is non-increasing in reconstruction error. This suggests replacing gradient updates for low-rank neural layers or adapters with streaming TT-ALS updates on minibatches of activation-target pairs, while using orthogonal cores to improve conditioning and prevent factor scale drift. The most credible first target is a frozen-feature linear or MLP projection, where the local least-squares problems are explicit and can be compared directly against LoRA and AdamW.",
  "title": "Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 6,
  "usefulness": 8
 },
 "solves": [
  "speedup",
  "memory",
  "stability",
  "sample-efficiency"
 ],
 "title": "Streaming TT-ALS adapter",
 "url": "https://synthcore.org/idea/272/streaming-tt-als-adapter",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}