{
 "artifacts": null,
 "category": "dynamics",
 "description": "Partition neural-network parameters into blocks and update each block using a stochastic proximal best response, followed by Krasnoselskii relaxation. The relaxation factor and minibatch size become explicit stability knobs: aggressive stochastic updates are damped, while larger batches are used when the estimated update variance approaches the mean-square stability boundary.",
 "formulas_latex": [
  "$$\\widehat{T}_n^k=\\mathop{\\arg\\min}_{u}\\left\\{\\widehat{f}_n^k(u;w_{-n}^k)+\\frac{1}{2r_n}\\|u-w_n^k\\|^2\\right\\},\\qquad \\widehat{f}_n^k(u;w_{-n}^k)=\\frac{1}{B_k}\\sum_{b=1}^{B_k}f_n(u,w_{-n}^k;\\xi_{n,b}^k).$$",
  "$$w^{k+1}=(1-\\alpha_k)w^k+\\alpha_k\\widehat{T}^k,\\qquad \\rho_k=1-\\alpha_k(1-q),\\qquad \\mathbb{E}[\\|w^{k+1}-w^\\star\\|^2\\mid\\mathcal{F}_k]\\leq \\rho_k^2\\|w^k-w^\\star\\|^2+\\frac{\\alpha_k^2\\sigma_k^2}{B_k}.$$",
  "$$\\mathbb{E}[J_k^\\top P J_k]-P\\preceq-\\mu P,\\qquad P\\succ0,\\ \\mu\u003e0.$$",
  "$$B_k\\geq\\frac{\\alpha_k^2\\sigma_k^2}{(1-\\rho_k^2)\\varepsilon^2}.$$"
 ],
 "id": 2807,
 "implementation": "1. Integration point: replace the standard optimizer update in a small or medium neural network with blockwise updates. Blocks can be attention projections, MLP layers, convolutional filters, or expert parameters. For each block, compute a minibatch loss while holding the other blocks fixed, take one or a few proximal-gradient steps toward the block response, and then apply global Krasnoselskii relaxation. A cheap approximation to the proximal response is T_hat_n=w_n-r_n g_hat_n; use the exact proximal subproblem only for small blocks. 2. Pseudocode: sample a minibatch of size B_k; compute each block gradient g_hat_n; set T_hat_n=w_n-r_n g_hat_n; form w_n \u003c- (1-alpha_k)w_n+alpha_k T_hat_n. Estimate gradient-noise variance from two independent half-minibatches, sigma_hat_k^2, and increase B_k or decrease alpha_k when alpha_k^2 sigma_hat_k^2/B_k exceeds a chosen fraction of (1-rho_hat_k^2)||w-w_ema||^2. 3. Computed from the paper's mechanism: proximal response, variable sample size, relaxation, and mean-square Lyapunov recursion. Estimated empirically: contraction q, noise variance sigma_k^2, Jacobian J_k, and optional matrix P. Estimate q from ratios of response differences on two nearby parameter vectors, and solve a small semidefinite program for P using sampled low-dimensional block Jacobians. 4. First experiment: train a 3-layer MLP on MNIST and a small Transformer on Shakespeare, comparing Adam, unrelaxed proximal-gradient, and this method at matched examples processed. Sweep alpha and B. The predicted signature is a sharp rise in long-run parameter variance and occasional loss divergence when rho^2+alpha^2 sigma^2/(B||w-w^star||^2) approaches 1. The empirical stability boundary should move toward smaller alpha as B decreases, approximately following B proportional to alpha^2 divided by 1-[1-alpha(1-q)]^2, with qualitative agreement and preferably 20 percent relative error.",
 "math_summary": "Let w=(w_1,...,w_N) be parameter blocks, f_n(w_n,w_{-n};xi_n) the stochastic loss for block n, and r_n\u003e0 its proximal coefficient. A stochastic proximal best response is T_hat_n^k=argmin_u {f_hat_n^k(u;w_{-n}^k)+(2r_n)^(-1)||u-w_n^k||^2}, where f_hat_n^k is computed from a minibatch. The relaxed update is w^{k+1}=(1-alpha_k)w^k+alpha_k T_hat^k, with 0\u003calpha_k\u003c=1. If the exact response operator T is mean-square contractive, E[||T(w,xi)-T(v,xi)||^2] \u003c= q^2||w-v||^2 for q\u003c1, and the response noise has conditional variance at most sigma_k^2/B_k, then the relaxed recursion has deterministic contraction factor rho_k=1-alpha_k(1-q) and noise amplification alpha_k^2 sigma_k^2/B_k. A quadratic Lyapunov function V_k=||w^k-w^star||_P^2, with P positive definite, certifies stability when E[J_k^T P J_k]-P \u003c= -mu P for some mu\u003e0, where J_k is the local Jacobian of the relaxed update.",
 "math_tags": [
  "control-theory",
  "dynamical-systems",
  "optimization",
  "linear-algebra",
  "stochastic-processes"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.29130",
  "arxiv_url": "https://arxiv.org/abs/2608.29130",
  "summary_what_math_gives_to_ml": "The paper offers a transferable decentralized optimization mechanism: stochastic proximal best responses are combined with Krasnoselskii relaxation and variable sample sizes, with mean-square convergence to a unique Nash equilibrium under dynamic-stability conditions. Its most useful neural-network interpretation is a block-coordinate optimizer in which parameter groups perform approximate proximal responses to stochastic losses, while relaxation controls the gain of the coupled update. The LMI-based quadratic construction also suggests learning or estimating a common quadratic Lyapunov certificate for the optimizer, yielding a measurable stability boundary rather than relying only on validation loss.",
  "title": "A Systematic Approach to Mechanism Design with Stochastic Dynamic Stability",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "sample-efficiency"
 ],
 "title": "Mean-Square Proximal Relaxation Optimizer",
 "url": "https://synthcore.org/idea/2807/mean-square-proximal-relaxation-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}