{
 "artifacts": null,
 "category": "dynamics",
 "description": "Build a recurrent or state-space network with time-dependent transition parameters, but train it to forget perturbations at a common exponential rate across all admissible parameter schedules. The model should retain task-relevant long-term signals while suppressing dependence on arbitrary initial hidden states, reducing instability under changing inputs, curricula, or deployment-time dynamics.",
 "formulas_latex": [
  "\\left\\|(T_{1,n})_*\\mu-(T_{1,n})_*\\nu\\right\\|_{\\mathrm{TV}}\\le C s^n,\\qquad T_{1,n}=T_n\\circ\\cdots\\circ T_1,\\quad 0\u003cs\u003c1.",
  "h_{t+1}=F_{\\theta_t}(h_t,x_t),\\qquad D(P_n,Q_n)\\le C\\rho^n,\\quad 0\u003c\\rho\u003c1.",
  "\\mathcal{L}_{\\mathrm{mix}}=\\sum_{k=1}^{K}\\left[\\max\\left(0,\\log D_k-\\log C+k\\log\\rho\\right)\\right]^2.",
  "\\widehat\\rho=\\exp\\left(\\frac{1}{b-a}\\sum_{k=a+1}^{b}\\log\\frac{\\widehat D_k}{\\widehat D_{k-1}}\\right),\\qquad \\widehat D_k=D(P_k,Q_k)."
 ],
 "id": 2808,
 "implementation": "Integrate this into a GRU, diagonal SSM, or small residual RNN by making the transition parameters $\\theta_t$ depend on the input, layer, training step, or an externally specified schedule. For every minibatch, duplicate each sequence with two hidden-state initializations: $h_0^{(1)}$ and $h_0^{(2)}$, sampled from a broad fixed distribution or set to two deterministic points. Run both copies with identical inputs and compute $\\widehat D_k$ after every $k$-step block. A cheap choice is the average absolute difference across hidden coordinates; a stronger choice is Wasserstein-1 estimated by one-dimensional random projections. Add $\\mathcal L_{\\mathrm{mix}}$ above, using target envelope $C\\rho^k$, and optionally clip or reject transition updates that increase the estimated slope. Pseudocode: initialize two states; for $t=1,\\ldots,L$, compute both recurrent updates with the same $x_t$; every $k$ steps compute $D_k$; accumulate the hinge penalty; backpropagate task loss plus $\\lambda\\mathcal L_{\\mathrm{mix}}$. The paper supplies the mechanism and exponential form, but not neural-network constants $C$ or $\\rho$; estimate them from a held-out set of schedules by fitting the slope of $\\log \\widehat D_k$. First experiment: compare a GRU baseline and the constrained GRU on sequential MNIST or permuted sequential MNIST under random time-varying input-gate biases. Use hidden-state initializations separated by a fixed norm, measure $\\widehat D_k$ for 100 steps, and test schedules not seen during training. The quantitative prediction is a near-linear plot of $\\log \\widehat D_k$ versus $k$ with slope $\\log\\rho\u003c0$, and similar slopes across schedules; fitted slopes should vary by less than 20% and the distance should fall below $C\\rho^k$ after the penalty is active. An ablation without the mixing loss should show schedule-dependent decay or occasional growth.",
 "math_summary": "The paper proves that for a nonstationary composition $T_{1,n}=T_n\\circ\\cdots\\circ T_1$ of admissible heterochaos maps, two absolutely continuous initial measures $\\mu$ and $\\nu$ with Hölder densities satisfy $\\|(T_{1,n})_*\\mu-(T_{1,n})_*\\nu\\|_{\\mathrm{TV}}\\le C s^n$ for constants $C\u003e0$ and $s\\in(0,1)$ independent of the admissible parameter sequence. Here $(T_{1,n})_*$ is pushforward and $\\|\\cdot\\|_{\\mathrm{TV}}$ is total variation distance. Adapt this as a distributional forgetting condition for hidden states $h_{t+1}=F_{\\theta_t}(h_t,x_t)$: for two initial hidden-state distributions $P_0,Q_0$ exposed to the same input sequence, require $D(P_n,Q_n)\\le C\\rho^n$, where $D$ is an empirical Wasserstein-1 or random-projection distance and $\\rho\\in(0,1)$. The useful property is a schedule-independent decay envelope, not merely contraction for one fixed transition.",
 "math_tags": [
  "dynamical-systems",
  "probability",
  "statistics",
  "control-theory"
 ],
 "ml_areas": [
  "rnn",
  "ssm",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.29135",
  "arxiv_url": "https://arxiv.org/abs/2608.29135",
  "summary_what_math_gives_to_ml": "The paper provides a uniform statistical-stability mechanism for deterministic, nonstationary compositions of heterochaos baker maps: despite arbitrary admissible parameter variation, sufficiently regular initial distributions lose memory exponentially fast. Its transferable asset is not the specific baker map, but the construction of a time-varying dynamical system with a sequence-independent mixing rate and a corresponding central-limit approximation. A practical neural analogue is a recurrent or state-space architecture whose transition parameters change during training or across inference time, while an explicit forgetting monitor and constraint enforce a common decay envelope. The key falsifiable prediction is that hidden-state distributions initialized differently should converge at an approximately exponential rate even under parameter schedules unseen during training.",
  "title": "Decay of correlations and normal approximation for nonstationary heterochaos baker maps",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization"
 ],
 "title": "Uniformly Mixing Nonstationary State-Space Network",
 "url": "https://synthcore.org/idea/2808/uniformly-mixing-nonstationary-state-space-network",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}