arXiv:2609.03856 2026 Architecture 2 ideas extracted · analyzed Sep 4, 2026
The paper studies an asynchronous nonlinear consensus operator in which one graph vertex at a time is replaced by the unique minimizer of its incident ℓ^p energy. The transferable asset is a tunable p-mean message-passing rule with explicit dependence of consensus time on graph geometry: conductance expanders mix in Θ(n log n) updates, while boxes and poorly connected graphs exhibit polynomial slowdowns. This suggests sparse token-mixing layers and decentralized synchronization modules that replace dense averaging or attention with local nonlinear updates, while using p and the graph schedule to trade robustness, expressivity, and computation.
Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.
Add asynchronous nonlinear consensus steps between local-SGD updates in a federated or decentralized system. Instead of averaging a participating client with all peers, replace one client parameter vector by the coordinatewise p-mean of neighboring replicas, allowing sparse communication and reduced sensitivity to atypical client models.