{
 "artifacts": null,
 "category": "architecture",
 "description": "Use exponentially weighted states as slowly moving prototypes for online mixture-of-experts routing or continual learning. Route each embedding by its distance to the current prototype, then update only the selected prototype; the memory parameter controls the tradeoff between rapid adaptation and resistance to routing noise.",
 "formulas_latex": [
  "$$x_i=\\gamma x_{i-1}+(1-\\gamma)p_i,\\qquad x_i\\in B_2^d(1)\\ \\text{if}\\ p_i\\in B_2^d(1).$$",
  "$$\\ell_i=\\left\\|p_i-x_{i-1}\\right\\|^\\alpha,\\qquad \\overline{\\ell}_N=\\frac{1}{N}\\sum_{i=1}^N\\ell_i.$$",
  "$$\\sup_{(p_i)}\\limsup_{N\\to\\infty}\\overline{\\ell}_N=\\left(\\frac{2}{1+\\gamma}\\right)^\\alpha,\\qquad 0\u003c\\alpha\\leq3.$$",
  "$$\\gamma_N^*=1-N^{-1/2}+\\frac{1}{2}N^{-1}+O(N^{-3/2})\\qquad(\\alpha=2).$$"
 ],
 "id": 170,
 "implementation": "Add one prototype state x_e per expert e in a small MoE layer or per class in a continual-learning classifier. At step t, compute a normalized embedding p_t=normalize(W_p h_t). For each expert, compute the score s_{t,e}=-||p_t-x_{e}||_2^2/tau, where tau is a learned or fixed routing temperature, and select the top one or top two experts. After the selected expert processes the example, update only its state using x_e=gamma_e x_e+(1-gamma_e)p_t; leave unselected states unchanged. Track an exponentially averaged assignment count m_e and set an estimated horizon N_e=max(1,total_steps/(m_e+epsilon)); initialize gamma_e=1-1/sqrt(N_e)+0.5/N_e, clipping gamma_e to [0,0.9999]. Normalize or clip embeddings before routing because the paper's guarantees assume bounded points. Compute the theoretical robustness proxy B_e=(2/(1+gamma_e))^2, while estimating the actual mean attachment loss from minibatches. The first cheap experiment should use a 4- or 8-expert Transformer on CIFAR-100 or a streaming language-model benchmark. Compare learned router logits, random routing, fixed EMA prototypes, and the horizon-tuned prototypes. Track validation accuracy or perplexity, expert-load imbalance, routing-switch frequency, average and worst-decile attachment loss, and performance after adversarially ordering classes or topics. The hypothesis is fewer unstable routing switches and better recovery under distribution shifts at comparable FLOPs.",
 "math_summary": "Let p_i be normalized embeddings in B_2^d(1), and let x_{i-1} be a prototype before observing p_i. The paper's attachment penalty is l_i=||p_i-x_{i-1}||^alpha, where alpha\u003e0, and the prototype update is x_i=gamma x_{i-1}+(1-gamma)p_i. Because this is a convex combination, x_i remains in the unit ball whenever p_i does. The endpoint gamma=0 follows the current sequence locally, while gamma=1 freezes the initial anchor. For d\u003e=2, the distributionally optimal fixed parameter for N observations has 1-gamma_N^*=Theta(N^{-1/2}); for alpha=2, the stated expansion is gamma_N^*=1-N^{-1/2}+1/(2N)+O(N^{-3/2}). For arbitrary sequences and 0\u003calpha\u003c=3, the asymptotic worst-case mean attachment cost is (2/(1+gamma))^alpha. The neural adaptation uses squared distances, alpha=2, as routing scores and uses the horizon estimate N_e for each expert to select gamma.",
 "math_tags": [
  "geometry",
  "probability",
  "optimization",
  "dynamical-systems"
 ],
 "ml_areas": [
  "moe-routing",
  "embedding",
  "continual-learning"
 ],
 "paper": {
  "arxiv_id": "2608.25298",
  "arxiv_url": "https://arxiv.org/abs/2608.25298",
  "summary_what_math_gives_to_ml": "The paper studies a one-vector online state that interpolates between retaining only the previous observation and retaining a persistent anchor. Its transferable asset is the explicit geometric control of attachment distances: exponential memory remains a convex combination of bounded observations and improves the worst-case movement scale while using constant working memory. This suggests compressing streaming neural representations into a single prototype used as global context, retrieval state, or a routing anchor. The paper also provides a horizon-dependent tuning law, including an explicit asymptotic expansion for squared distance, making the proposal directly testable.",
  "title": "Sequential Euclidean tree construction with exponential memory: distributional performance and worst-case guarantees",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 5,
  "usefulness": 5
 },
 "solves": [
  "accuracy",
  "stability",
  "sample-efficiency"
 ],
 "title": "Horizon-Tuned Prototype Router",
 "url": "https://synthcore.org/idea/170/horizon-tuned-prototype-router",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}