{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.",
 "formulas_latex": [
  "$$\\mathsf{T}_{p}(G,1/2):=\\sup_{\\lVert f_{0}\\rVert_{\\infty}\\le1}\\mathbb{E}\\left[\\min\\{t\\ge0:\\operatorname{osc}(f_t)\\le1/2\\}\\right].$$",
  "$$m_p(v)=\\arg\\min_{a\\in\\mathbb{R}}\\sum_{u\\sim v}|a-f(u)|^p,\\qquad \\sum_{u\\sim v}|m_p(v)-f(u)|^{p-2}(m_p(v)-f(u))=0.$$",
  "$$f_{t+1}(v)=m_p(v;f_t),\\qquad f_{t+1}(u)=f_t(u)\\quad(u\\neq v),\\quad v\\sim\\operatorname{Uniform}(V).$$",
  "$$\\mathsf{T}_{p}([L]^d,1/2)\\asymp_{\\log}\\begin{cases}nd^{1/(p-1)}L^{p/(p-1)},\u00261\u003cp\u003c2,\\\\ndL^2,\u0026p\\ge2,\\end{cases}\\qquad n=L^d.$$"
 ],
 "id": 3152,
 "implementation": "1. Integration point: insert the module between Transformer blocks as a token-mixing residual layer. Construct a fixed sparse graph on the T token positions, such as a 2-D grid for image patches, a ring plus local windows for text, or a k-nearest-neighbor graph in positional space. Apply the same operation independently to every hidden channel, optionally after a learned scalar gate and normalization. 2. Pseudocode: input X[T,d]; set Y=X; for k=1,...,K choose vertex v using a random, cyclic, or learned schedule; for every channel c solve g(a)=sum_{u in N(v)} |a-Y[u,c]|^(p-2)(a-Y[u,c])=0 by 8-12 bisection iterations on the interval [min_u Y[u,c], max_u Y[u,c]]; set Y[v,c]=a and leave all other rows unchanged; return X+alpha*(Y-X). Bisection is valid because g is monotone for p\u003e1 and the minimizer lies in the neighbor-value interval. 3. Compute the p-mean numerically, while treating the paper's consensus-time scaling as a qualitative graph-design prediction rather than an exact neural-network guarantee. For differentiable implementation, backpropagate through the fixed bisection iterations, or use the implicit derivative da/dy_j=|a-y_j|^(p-2)/sum_l |a-y_l|^(p-2) away from coincident values. Track the token oscillation statistic osc(Y) and the norm of the residual update. 4. First cheap experiment: compare a 6-layer small Transformer using this mixer against local-window attention and full attention on CIFAR-10 ViT-Tiny or WikiText-2 with equal hidden size and matched training FLOPs. Sweep p in {1.25,1.5,2,3,4}, K, and alpha. Success means lower validation loss or higher accuracy than local averaging at equal FLOPs, with token-mixing cost scaling as O(KTd) rather than O(T^2d), and no instability in the oscillation statistic.",
 "math_summary": "The paper defines the asynchronous relaxation time as \\(\\mathsf{T}_{p}(G,1/2)=\\sup_{\\lVert f_{0}\\rVert_{\\infty}\\le1}\\mathbb{E}[\\min\\{t\\ge0:\\operatorname{osc}(f_t)\\le1/2\\}]\\), where \\(G=(V,E)\\) has \\(n=|V|\\) vertices, \\(f_t:V\\to\\mathbb{R}\\) is the current profile, and \\(\\operatorname{osc}(f)=\\max_v f(v)-\\min_v f(v)\\). For a selected vertex \\(v\\), the local update minimizes \\(\\Phi_v(a)=\\sum_{u\\sim v}|a-f(u)|^p\\), with \\(1\u003cp\u003c\\infty\\). Strict convexity gives a unique minimizer \\(m_p(v)\\), characterized by \\(\\sum_{u\\sim v}|m_p(v)-f(u)|^{p-2}(m_p(v)-f(u))=0\\). The paper reports \\(\\mathsf{T}_p([L]^d,1/2)\\) up to logarithmic factors as \\(nd^{1/(p-1)}L^{p/(p-1)}\\) for \\(1\u003cp\u003c2\\), and \\(ndL^2\\) for \\(p\\ge2\\), while conductance expanders with \\(h(G)\\ge h_0\u003e0\\) have \\(\\mathsf{T}_p(G,1/2)=\\Theta_{p,h_0}(n\\log n)\\). The proposed neural adaptation uses the local minimization rule and graph-dependent mixing behavior to build a sparse token mixer.",
 "math_tags": [
  "graph-theory",
  "optimization",
  "convex-analysis",
  "dynamical-systems"
 ],
 "ml_areas": [
  "attention",
  "transformer",
  "graph-nn",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2609.03856",
  "arxiv_url": "https://arxiv.org/abs/2609.03856",
  "summary_what_math_gives_to_ml": "The paper studies an asynchronous nonlinear consensus operator in which one graph vertex at a time is replaced by the unique minimizer of its incident ℓ^p energy. The transferable asset is a tunable p-mean message-passing rule with explicit dependence of consensus time on graph geometry: conductance expanders mix in Θ(n log n) updates, while boxes and poorly connected graphs exhibit polynomial slowdowns. This suggests sparse token-mixing layers and decentralized synchronization modules that replace dense averaging or attention with local nonlinear updates, while using p and the graph schedule to trade robustness, expressivity, and computation.",
  "title": "Consensus time for asynchronous $\\ell^p$ relaxation: graph dependence",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "accuracy",
  "scalability"
 ],
 "title": "Asynchronous p-Mean Token Mixer",
 "url": "https://synthcore.org/idea/3152/asynchronous-p-mean-token-mixer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}