{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace ordinary global gradient clipping with clipping of each stochastic gradient around a robust minibatch center rather than around zero. This preserves the common directional component of the gradients and suppresses only heavy-tailed residuals, making the update usable when gradient noise has a finite α-moment for 1 \u003c α ≤ 2 but no finite variance.",
 "formulas_latex": [
  "$$\\mathbb{E}\\!\\left[\\|G_t-\\nabla f(x_t)\\|^{\\alpha}\\mid\\mathcal{F}_{t-1}\\right]\\leq\\sigma^{\\alpha},\\qquad 1\u003c\\alpha\\leq 2.$$",
  "$$\\frac{1}{2}\\|\\nabla f(x)\\|^{2}\\geq\\mu\\bigl(f(x)-f^{\\star}\\bigr).$$",
  "$$T_{\\epsilon}=\\Omega_{\\alpha}\\!\\left[\\kappa\\log\\frac{\\Delta_{0}}{\\epsilon}+\\kappa\\left(\\frac{\\sigma^{2}}{\\mu\\epsilon}\\right)^{\\frac{\\alpha}{2(\\alpha-1)}}\\right].$$",
  "$$C_{\\tau}(g;c)=c+(g-c)\\min\\left\\{1,\\frac{\\tau}{\\|g-c\\|}\\right\\},\\qquad x_{t+1}=\\arg\\min_x\\left\\{\\langle h_t,x\\rangle+\\frac{1}{\\eta_t}D_{\\psi}(x,x_t)\\right\\}.$$"
 ],
 "id": 3159,
 "implementation": "Integrate this at the optimizer update, after per-example gradients are computed and before momentum or parameter updates. For each minibatch B_t, let g_i be the flattened gradient of example i. Compute a robust center c_t using the coordinate-wise median for a cheap baseline, or approximate the geometric median with 3-5 Weiszfeld iterations. Form residuals r_i=g_i-c_t and clip them with h_i=c_t+r_i*min(1,tau/(||r_i||_2+1e-12)). Average h_t=mean_i h_i, optionally apply momentum, and update parameters with Euclidean mirror descent theta=theta-eta*h_t. For Adam compatibility, feed h_t rather than the raw batch gradient into the moment accumulators: m=beta1*m+(1-beta1)*h_t; v=beta2*v+(1-beta2)*h_t*h_t; theta=theta-eta*m/(sqrt(v)+eps). Initialize tau=k*sigma_hat*(|B_t|/log(2/delta))^(1/alpha), with k in {1,2,4}; estimate sigma_hat from the median or 0.8-quantile of residual norms using a slow EMA. If alpha is unknown, sweep alpha in {1.2,1.5,1.8,2}. The paper supplies the finite-moment assumption, PL contraction structure, and target complexity; the center, scale estimator, and layerwise radius are empirical approximations. First test a 2-layer MLP or small ResNet on CIFAR-10 with Student-t gradient perturbations of degrees of freedom 1.2-1.8 and occasional 10-100x outliers. Compare SGD, global norm clipping, AdamW, and this optimizer at equal minibatch size and FLOPs. Measure loss descent, update-norm quantiles, divergence frequency, validation accuracy, and robustness as tails worsen. Success means fewer catastrophic steps and lower loss or higher accuracy at equal compute, while matching standard optimizers on Gaussian-noise controls.",
 "math_summary": "The paper assumes conditionally unbiased oracle gradients G_t with E[G_t | F_{t-1}] = ∇f(x_t) and finite conditional moment E[||G_t - ∇f(x_t)||^α | F_{t-1}] ≤ σ^α, where F_{t-1} is the history before step t, α ∈ (1,2], and σ is the heavy-tail scale. The objective obeys the PL inequality (1/2)||∇f(x)||^2 ≥ μ(f(x)-f★), where μ is the PL constant and f★ is the global minimum value; L-smoothness gives condition number κ=L/μ. The paper's target stochastic complexity is T_ε = Ω_α[κ log(Δ_0/ε) + κ(σ^2/(με))^{α/[2(α-1)]}], with Δ_0=f(x_0)-f★. Use the centered clipping operator C_τ(g;c)=c+(g-c)min{1,τ/||g-c||}, where c is a robust estimate of the current mean gradient and τ is a residual radius. For a mirror map ψ, use x_{t+1}=argmin_x{\u003ch_t,x\u003e+(1/η_t)D_ψ(x,x_t)}, where h_t is the averaged centered-clipped gradient and D_ψ(x,y)=ψ(x)-ψ(y)-\u003c∇ψ(y),x-y\u003e is the Bregman divergence. In Euclidean parameter space this reduces to x_{t+1}=x_t-η_t h_t. Clipping bias is controlled by the finite moment because E[||(G-∇f)1_{||G-∇f||\u003eτ}||] ≤ σ^α/τ^{α-1}; increasing τ therefore trades bias against outlier sensitivity.",
 "math_tags": [
  "optimization",
  "probability",
  "convex-analysis",
  "dynamical-systems"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2609.03990",
  "arxiv_url": "https://arxiv.org/abs/2609.03990",
  "summary_what_math_gives_to_ml": "The paper gives a principled route for training under finite-α-moment gradient noise, where variance may be infinite and conventional sub-Gaussian concentration or unclipped SGD guarantees fail. Its transferable asset is centered clipping combined with mirror descent: clipping controls rare, arbitrarily large stochastic gradients while preserving the common directional component of the gradients. A practical neural-network adaptation is a robust optimizer that estimates a local gradient center from a minibatch, clips residuals around that center, and performs an Adam-like or mirror-descent update. The main falsifiable target is improved tail robustness and convergence when gradients are contaminated by heavy-tailed noise.",
  "title": "Heavy-Tailed First-Order Optimization for Polyak-Łojasiewicz Condition: High-Dimensional Minimax Bounds, High-Probability Guarantee, and Fixed-Dimensional Improvements",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "speedup",
  "generalization"
 ],
 "title": "Centered Heavy-Tail Clipping Optimizer",
 "url": "https://synthcore.org/idea/3159/centered-heavy-tail-clipping-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}