{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace repeated nonlinear optimization of a small neural module with a predetermined random-feature basis and a constrained linear output solve. After the least-squares solution, project the output coefficients onto an exactly feasible affine set so that a chosen linear invariant is satisfied despite finite residual error.",
 "formulas_latex": [
  "$$u_{\\mathrm{NN}}(\\bm{x},t)=\\sum_{m=1}^{N}\\alpha_m(t)\\,\\tanh(\\bm{w}_m^{T}\\bm{x}+b_m),$$",
  "$$\\hat{\\alpha}=\\arg\\min_{\\alpha\\in\\mathbb{R}^{N}}\\|\\Phi\\alpha-y\\|_2^2,$$",
  "$$\\alpha^*=\\arg\\min_{\\alpha}\\|\\alpha-\\hat\\alpha\\|_2^2\\quad\\mathrm{s.t.}\\quad c^T\\alpha=m\\;=\\;\\hat\\alpha+c(c^Tc)^{-1}(m-c^T\\hat\\alpha),$$",
  "$$c_m=\\sum_{i=1}^{K}q_i\\Phi_{im},\\qquad q^T(\\Phi\\alpha^*)=m.$$"
 ],
 "id": 119,
 "implementation": "Integrate this at the output layer of a compact MLP used as a recurrent state updater, neural operator, or adapter, rather than replacing a full transformer. Inputs are sample coordinates, graph nodes, physical states, or latent positions $x_i$. Choose $N$ fixed hidden features $\\phi_m(x)=\\tanh(w_m^Tx+b_m)$ and construct $\\Phi_{im}=\\phi_m(x_i)$ once. Choose a known linear invariant represented by aggregation or quadrature weights $q\\in\\mathbb{R}^K$; for mass conservation, set $m=q^Ty_{\\mathrm{initial}}$. Precompute $c=\\Phi^Tq$, factor $\\Phi^T\\Phi+\\lambda I$, and optionally cache its factorization. At each update, form the target $y_t$ from the baseline state update, solve $\\hat\\alpha=(\\Phi^T\\Phi+\\lambda I)^{-1}\\Phi^Ty_t$, then execute `err=m-dot(c,hat_alpha); alpha=hat_alpha+c*err/(dot(c,c)+1e-12)`. Return $\\Phi\\alpha$ as the output. For multiple invariants, stack rows into $C\\in\\mathbb{R}^{r\\times N}$ and use `alpha=hat_alpha+C.T@solve(C@C.T,b-C@hat_alpha)`. Compute feature conditioning empirically from the singular values of $\\Phi$; compute $c$ from known quadrature weights rather than learning it. First test a 2D Allen-Cahn or advection rollout and a graph diffusion task with a 2-layer, 256-feature baseline trained by Adam. Compare against full-network SGD and unconstrained random features at equal parameter count and rollout horizon. Measure wall-clock time, peak memory, task MSE, rollout stability, and invariant drift. Success means at least 3x faster output updates, invariant error near numerical precision, and no more than 5% accuracy loss. Also test transfer by reusing the frozen features on a second domain or initial condition without retraining hidden layers.",
 "math_summary": "The paper defines the transferable representation $u_{\\mathrm{NN}}(\\bm{x},t)=\\sum_{m=1}^{N}\\alpha_m(t)\\tanh(\\bm{w}_m^T\\bm{x}+b_m)$, where $\\bm{x}$ is the input, $N$ is the number of hidden neurons, $(\\bm{w}_m,b_m)$ are fixed hidden parameters, and $\\alpha_m(t)$ are time-dependent output coefficients. For samples $x_i$, define the fixed feature matrix $\\Phi\\in\\mathbb{R}^{K\\times N}$ by $\\Phi_{im}=\\tanh(\\bm{w}_m^Tx_i+b_m)$. Given a target vector $y\\in\\mathbb{R}^{K}$, compute the unconstrained least-squares coefficient $\\hat{\\alpha}=\\arg\\min_{\\alpha}\\|\\Phi\\alpha-y\\|_2^2$. Let $c\\in\\mathbb{R}^{N}$ encode a linear invariant and let $m$ be its desired value, so feasibility is $c^T\\alpha=m$. The minimum-change Euclidean projection is $\\alpha^*=\\arg\\min_\\alpha\\|\\alpha-\\hat\\alpha\\|_2^2$ subject to $c^T\\alpha=m$, with closed form $\\alpha^*=\\hat\\alpha+c(c^Tc)^{-1}(m-c^T\\hat\\alpha)$. If the invariant is defined on sampled outputs using quadrature weights $q\\in\\mathbb{R}^{K}$, then $c=\\Phi^Tq$ and $c^T\\alpha=q^T(\\Phi\\alpha)$. For a weighted coefficient metric $W\\succ0$, the corresponding projection is $\\alpha^*=\\hat\\alpha+W^{-1}c(c^TW^{-1}c)^{-1}(m-c^T\\hat\\alpha)$.",
 "math_tags": [
  "linear-algebra",
  "optimization",
  "numerical-analysis"
 ],
 "ml_areas": [
  "mlp",
  "optimizer",
  "training",
  "loss"
 ],
 "paper": {
  "arxiv_id": "2608.23980",
  "arxiv_url": "https://arxiv.org/abs/2608.23980",
  "summary_what_math_gives_to_ml": "The paper's most transferable machinery is a fixed-feature neural representation in which all nonlinear hidden parameters are sampled once and only a linear output layer is recomputed at each time step. This converts repeated nonlinear optimization into a constant-design-matrix least-squares solve, making the representation cheap to update and naturally reusable across time, domains, or related tasks. Its stronger contribution is an explicit post-solve projection onto a linear invariant, which can be generalized from Cahn-Hilliard mass conservation to exact conservation of token counts, graph mass, probability normalization, or other known linear constraints in neural outputs.",
  "title": "Structure-preserving generalized transferable neural networks for the Cahn-Hilliard equation",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "stability",
  "memory"
 ],
 "title": "Fixed-feature invariant output layer",
 "url": "https://synthcore.org/idea/119/fixed-feature-invariant-output-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}