{
 "artifacts": [
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1034/experiment.py"
  },
  {
   "name": "fractional_memory.py",
   "url": "https://synthcore.org/code/1034/fractional_memory.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1034/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1034/results.json"
  },
  {
   "name": "verify.py",
   "url": "https://synthcore.org/code/1034/verify.py"
  }
 ],
 "category": "architecture",
 "description": "Construct an efficient recurrent or state-space layer whose impulse response follows Mittag-Leffler relaxation instead of a single exponential. A bank of stable diagonal state channels approximates the long power-law tail, allowing the layer to retain information over widely separated timescales with only \\(K\\) states per feature.",
 "download_zip": "https://synthcore.org/download/1034",
 "formulas_latex": [
  "$$e_{\\alpha}(t):=E_{\\alpha}(-t^{\\alpha}):=\\sum_{n=0}^{\\infty}\\frac{(-t^{\\alpha})^{n}}{\\Gamma(\\alpha n+1)}$$",
  "$$e_{\\alpha}(t)\\sim \\frac{t^{-\\alpha}}{\\Gamma(1-\\alpha)},\\qquad t\\to\\infty,\\quad 0\u003c\\alpha\u003c1$$",
  "$$y_t=\\frac{\\sum_{j=1}^{t}h_{t-j}x_j}{\\sum_{\\ell=0}^{t-1}h_\\ell},\\qquad h_\\ell=E_{\\alpha}\\!\\left(-\\left(\\frac{\\ell+1}{\\tau}\\right)^{\\alpha}\\right)$$",
  "$$s_{t,k}=\\rho_ks_{t-1,k}+(1-\\rho_k)x_t,\\qquad y_t=\\sum_{k=1}^{K}w_ks_{t,k},\\qquad \\hat h_\\ell=\\sum_{k=1}^{K}w_k(1-\\rho_k)\\rho_k^\\ell$$"
 ],
 "id": 2677,
 "implementation": "Add the module as a drop-in replacement for the temporal mixing block in a small causal Transformer, GRU, or diagonal SSM. For an input sequence tensor \\(x\\in\\mathbb{R}^{T\\times B\\times d}\\), create \\(K\\) state tensors \\(s_k\\in\\mathbb{R}^{B\\times d}\\); use \\(K=8\\), positive weights normalized by softmax, and decay factors \\(\\rho_k=\\exp(-\\Delta/\\tau_k)\\) with logarithmically spaced \\(\\tau_k\\) from 1 to the target context length. At every timestep, compute `s[k] = rho[k] * s[k] + (1-rho[k]) * x[t]`, then `y[t] = sum(w[k] * s[k])`; apply a learned input/output projection and residual connection around the filter. Optionally learn \\(\\alpha\\) and \\(\\tau\\) only through a differentiable initialization map that refits \\(w_k,\\rho_k\\), but begin with fixed stable decays. Fit the bank before training by minimizing squared error between \\(\\hat h_\\ell\\) and the paper kernel \\(E_\\alpha(-((\\ell+1)/\\tau)^\\alpha)\\) over logarithmic lags; this fit is the empirical approximation, while the power-law asymptotic is the mathematical motivation. The first cheap experiment should compare this layer with a GRU, a single-exponential SSM, and a standard causal Transformer on synthetic delayed-copy and adding problems at sequence lengths 1k, 4k, and 16k, followed by TinyStories language modeling. Track accuracy as delay/context grows, validation perplexity at equal parameter count, and memory/runtime. Success is materially better delayed-copy accuracy and less perplexity degradation at long context with \\(O(KTd)\\) compute and constant recurrent state.",
 "math_summary": "The paper's temporal building block is \\(e_\\alpha(t)=E_\\alpha(-t^\\alpha)=\\sum_{n=0}^{\\infty}(-t^\\alpha)^n/\\Gamma(\\alpha n+1)\\), with fractional order \\(\\alpha\\) and time \\(t\\). Its stated asymptotic \\(e_\\alpha(t)\\sim t^{-\\alpha}/\\Gamma(1-\\alpha)\\) for \\(0\u003c\\alpha\u003c1\\) gives a slowly decaying memory kernel. For a discrete neural sequence \\(x_1,\\ldots,x_T\\), define the desired causal normalized filter \\(y_t=\\sum_{j=1}^{t}h_{t-j}x_j/\\sum_{\\ell=0}^{t-1}h_\\ell\\), where \\(h_\\ell=E_\\alpha(-((\\ell+1)/\\tau)^\\alpha)\\), \\(\\tau\u003e0\\) is a characteristic timescale, and \\(\\ell=t-j\\) is the lag. Implement it using a positive exponential mixture \\(h_\\ell\\approx\\hat h_\\ell=\\sum_{k=1}^{K}w_k(1-\\rho_k)\\rho_k^\\ell\\). Each component has state \\(s_{t,k}=\\rho_ks_{t-1,k}+(1-\\rho_k)x_t\\), and the output is \\(y_t=\\sum_kw_ks_{t,k}\\); stability follows from \\(0\u003c\\rho_k\u003c1\\), while logarithmically spaced \\(\\rho_k\\) approximate the fractional power-law memory.",
 "math_tags": [
  "fractional-calculus",
  "dynamical-systems",
  "approximation-theory",
  "numerical-analysis"
 ],
 "ml_areas": [
  "ssm",
  "rnn",
  "transformer",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2608.21674",
  "arxiv_url": "https://arxiv.org/abs/2608.21674",
  "summary_what_math_gives_to_ml": "The paper identifies Mittag-Leffler relaxation as a constructive alternative to exponential forgetting: for fractional order \\(0\u003c\\alpha\u003c1\\), its tail decays as a power law \\(t^{-\\alpha}\\), retaining information over many timescales. This is transferable to neural systems because standard exponential moving averages, recurrent states, and linear state-space models have geometrically decaying memory and therefore forget old signals too quickly. The most practical adaptation is to approximate a Mittag-Leffler kernel with a small bank of exponential states, preserving long memory while retaining \\(O(KT)\\) computation and constant-size recurrent state. This can be tested both as a multi-timescale optimizer accumulator and as a long-context recurrent or SSM layer.",
  "title": "Anomalous diffusion memory factorization: Characteristic timescales and application to inverse problem",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "scalability",
  "sample-efficiency"
 ],
 "title": "Fractional-memory recurrent state",
 "url": "https://synthcore.org/idea/2677/fractional-memory-recurrent-state",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a stable PyTorch fractional-memory layer using positive softmax-weighted exponential states with logarithmically spaced time constants, plus numerical verification scripts. The Mittag-Leffler power-law asymptotic matched the predicted 1/sqrt(pi) constant with relative error falling from 0.49% at lag 100 to 0.00005% at lag 1e6, and exponential stability/half-life predictions matched discrete observations within one step. However, bank approximation error improved strongly from K=1 to K=4 but then saturated rather than continuing to improve, and the fractional K=8 filter had higher random-signal MSE than the best single exponential; therefore the full claimed mechanism was not demonstrated as a win.",
   "metrics": {
    "baseline": "Best single exponential: MSE 0.0001200 against the exact normalized fractional filter; delayed impulse at lag 4000: 1.29e-58.",
    "idea": "Fractional K=8 bank: MSE 0.0006433; delayed impulse at lag 4000: 1.52e-05, approximately 1.17e53 times larger retention than the tau=32 baseline. Relative kernel-fit error by K: 1=101.99, 2=0.818, 4=0.338, 8=0.314, 16=0.313, 32=0.313."
   },
   "how_to_run": "python3 verify.py",
   "files": [
    "fractional_memory.py",
    "verify.py",
    "results.json"
   ],
   "limitations": "No trained delayed-copy model, GRU, Transformer, language-model, equal-parameter benchmark, GPU benchmark, or alpha values other than 1/2 were tested. The bank fit used fixed logarithmic time constants and a finite horizon, so improved fitting or learned decays remain unexplored."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-01T20:09:01",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}