{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace a dense learned polynomial-feature transform with a d-orthogonal recurrence whose production matrix is constrained to a (d+2)-banded lower-Hessenberg form. The layer generates successive features using only local recurrence coefficients, giving O(dN) arithmetic and O(dN) parameters for N basis functions instead of O(N^2) dense mixing.",
 "formulas_latex": [
  "$$p_n(x)=\\sum_{k=0}^{n}A_{n,k}x^k,\\qquad P_{A^{-1}}=ASA^{-1},\\qquad S_{n,k}=\\delta_{k,n+1}.$$",
  "$$x\\,p_n(x)=\\sum_{j=-d}^{1}P_{n,n-j}\\,p_{n-j}(x),\\qquad P_{n,k}=0\\ \\text{unless}\\ n-d-1\\le k\\le n+1.$$",
  "$$\\begin{aligned}q_0\u0026=1,\\\\ q_{n+1}\u0026=\\frac{1}{P_{n,n+1}}\\left(xq_n-\\sum_{j=0}^{d}P_{n,n-j}q_{n-j}\\right),\\qquad n\\ge 0,\\end{aligned}$$",
  "$$\\text{cost}_{\\mathrm{recurrence}}=O(dN)\\quad\\text{versus}\\quad \\text{cost}_{\\mathrm{dense\\ basis}}=O(N^2).$$"
 ],
 "id": 2794,
 "implementation": "(1) Integration point: insert the module between an encoder and an MLP head, or use it as a replacement for a dense polynomial-feature block. For an input vector u in R^D, first compute a scalar coordinate x=tanh(w^T u+b), with w normalized or spectrally constrained; optionally use one independent module per coordinate or per channel. The output is h=(q_0(x),...,q_{N-1}(x)), followed by a linear projection. Train the recurrence coefficients jointly with the rest of the network, but parameterize P so that only the diagonals k=n+1,n,n-1,...,n-d are stored. (2) Pseudocode: set q[0]=ones_like(x); for n=0,...,N-2, compute rhs=x*q[n]; subtract P[n,n-j]*q[n-j] for j=0,...,min(d,n); set q[n+1]=rhs/P[n,n+1]. Use softplus(P[n,n+1]) plus epsilon for a positive forward coefficient. Concatenate q or apply a learned projection. (3) The paper's mathematical ingredient is the sparse production-matrix/banded-recurrence correspondence. Estimate no matrix inverse during training; the recurrence coefficients are direct parameters. For stability, estimate the empirical second moment of each q_n and apply per-order normalization, or initialize with Chebyshev coefficients for the three-term case and small lower-band coefficients for d\u003e1. Clip or regularize the lower-band coefficient norm by adding lambda times the squared Frobenius norm of the stored bands. (4) First experiment: compare a dense polynomial feature layer and this recurrence layer on MNIST or CIFAR-10 with a small MLP, using N=32 and d in {1,2,4}, equal hidden width and matched parameter counts. Measure validation accuracy, wall-clock time, peak memory, gradient norm, and activation variance as N increases. Success means comparable or better accuracy with at least 2x lower layer parameters and visibly lower memory, or faster loss descent at equal parameter count; failure is exploding activations, strong order-dependent conditioning, or accuracy loss exceeding 1 percentage point.",
 "math_summary": "Let p_n(x)=\\sum_{k=0}^{n}A_{n,k}x^k be a polynomial basis and let A be its lower-triangular coefficient matrix. Let S be the up-shift matrix, with S_{n,k}=\\delta_{k,n+1}. The paper defines the production matrix associated with A^{-1} by P_{A^{-1}}=ASA^{-1}. For an ordinary orthogonal polynomial sequence, P is tridiagonal and encodes a three-term recurrence; for a d-orthogonal sequence, P is (d+2)-banded lower Hessenberg and encodes a (d+2)-term recurrence. In a neural layer, use the equivalent recurrence x p_n(x)=\\sum_{j=-d}^{1}P_{n,n-j}p_{n-j}(x), where coefficients outside valid indices are zero; the upper one-step term generates p_{n+1}, while the d lower terms couple only the previous d+1 features. Here x is a scalar feature or a normalized projection of a vector input, n is the polynomial order, P_{i,j} are trainable recurrence coefficients, d controls bandwidth, and N is the number of generated features. The lower-Hessenberg constraint is the transferable structure: it makes basis generation sparse and avoids explicitly forming A or A^{-1}.",
 "math_tags": [
  "linear-algebra",
  "combinatorics",
  "numerical-analysis"
 ],
 "ml_areas": [
  "mlp",
  "architecture",
  "initialization"
 ],
 "paper": {
  "arxiv_id": "2608.28834",
  "arxiv_url": "https://arxiv.org/abs/2608.28834",
  "summary_what_math_gives_to_ml": "The paper supplies a structured change-of-basis framework in which polynomial coefficient matrices are replaced by production matrices, with orthogonal-polynomial recurrences appearing as tridiagonal operators and d-orthogonal recurrences as (d+2)-banded lower-Hessenberg operators. This suggests neural layers that compute learned polynomial features through short recurrences rather than dense Vandermonde-like matrix multiplication, reducing parameter count and improving numerical conditioning. The most transferable asset is the equivalence between a basis transform and a sparse recurrence operator, together with the inverse-matrix moment interpretation that can initialize or regularize the learned basis.",
  "title": "Riordan array representation of recursive polynomial sequences, orthogonal polynomial sequences, and $d$-orthogonal polynomial sequences",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 5
 },
 "solves": [
  "speedup",
  "memory",
  "stability"
 ],
 "title": "Banded Production-Matrix Polynomial Layer",
 "url": "https://synthcore.org/idea/2794/banded-production-matrix-polynomial-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}