{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace a fixed PAGE refresh schedule with a restart policy selected from the PL condition-number regime. For well-conditioned objectives, use frequent full-gradient refreshes and short inner phases; for ill-conditioned objectives, use the conventional condition-number-scaled PAGE phase length. The goal is lower component-gradient cost to a target loss while retaining PAGE's low-variance updates.",
 "formulas_latex": [
  "$$\\operatorname{IFO}=O\\!\\left(n+\\frac{n\\log(\\Delta/\\varepsilon)}{1+\\log(\\sqrt n/\\kappa_{\\rm ms})}\\right),\\qquad 1\\le\\kappa_{\\rm ms}\\le\\sqrt n,$$",
  "$$\\operatorname{IFO}=O\\!\\left(n+\\kappa_{\\rm ms}\\sqrt n\\log(\\Delta/\\varepsilon)\\right),\\qquad \\kappa_{\\rm ms}\\ge\\sqrt n,$$",
  "$$v_t=\\begin{cases}\\nabla F(x_t),\u0026\\text{with probability }p,\\\\[2pt]\\nabla f_{i_t}(x_t)-\\nabla f_{i_t}(x_{t-1})+v_{t-1},\u0026\\text{with probability }1-p,\\end{cases}$$",
  "$$x_{t+1}=x_t-\\eta v_t.$$"
 ],
 "id": 2960,
 "implementation": "Integrate this into an optimizer for a model trained on a finite dataset, treating each example or minibatch shard as a component function. Maintain x_prev and v_prev. At the start of each restart epoch compute the exact full-data gradient v=∇F(x); inside the epoch, either refresh exactly with probability p or sample a minibatch B and set v_t=v_{t-1}+|B|^{-1}Σ_{i∈B}[∇f_i(x_t)-∇f_i(x_{t-1})]. Then update x_{t+1}=x_t−ηv_t. Every K steps estimate the local PL parameter from recent checkpoints: μ_hat=median(||g_j||²/(2 max(F_j−F_best,δ))). Estimate mean-squared smoothness using random probes u_j and minibatch gradients G: L_hat_ms²=median(||G(x+u_j)−G(x)||²/||u_j||²). Set κ_hat=L_hat_ms/max(μ_hat,δ). If κ_hat≤√n, select a short restart phase T from a grid {1,2,4,...,√n}; if κ_hat\u003e√n, select T from {√n,2√n,...,κ_hat√n}. Adjust p approximately as one full refresh per phase, while clipping η to a conservative smoothness-based bound such as 0.1/L_hat_ms. Use stop-gradient around all estimates so they do not alter model gradients. First test on a strongly convex quadratic with controlled κ, then CIFAR-10 ResNet-18 and a 50M-parameter Transformer. Compare AdamW, fixed-schedule PAGE, and adaptive PAGE at equal optimizer steps, component-gradient FLOPs, parameters, and memory. Pre-register: for κ\u003c√n, adaptive PAGE must reduce calls to a fixed objective gap by at least 20% versus fixed PAGE; for κ\u003e√n, its cost must be within 10% of fixed PAGE; and removing exact refreshes must increase measured estimator variance by at least 2x. Record loss-versus-component-calls, gradient variance, estimated κ, and selected T. Falsify the transfer if the low-κ advantage is absent on controlled quadratics or if adaptive scheduling increases high-κ cost by more than 10%.",
 "math_summary": "Let F(x)=n^{-1}\\sum_{i=1}^n f_i(x), where n is the number of component losses. Assume the global PL inequality \\|\\nabla F(x)\\|^2\\ge 2\\mu(F(x)-F^*) with \\mu\u003e0. Let L_{\\rm ms} be the mean-squared smoothness constant and define \\kappa_{\\rm ms}=L_{\\rm ms}/\\mu; alternatively, under individual smoothness use \\kappa_{\\max}=L_{\\max}/\\mu, where \\|\\nabla f_i(x)-\\nabla f_i(y)\\|\\le L_{\\max}\\|x-y\\| for every component i. The paper proves the restarted PAGE complexity bounds O(n+n\\log(\\Delta/\\varepsilon)/(1+\\log(\\sqrt n/\\kappa_{\\rm ms}))) for 1\\le\\kappa_{\\rm ms}\\le\\sqrt n and O(n+\\kappa_{\\rm ms}\\sqrt n\\log(\\Delta/\\varepsilon)) for \\kappa_{\\rm ms}\\ge\\sqrt n, where \\Delta=F(x_0)-F^* and \\varepsilon is the target suboptimality. PAGE stores a reference estimator v_t and refreshes it exactly with probability p; otherwise it uses a component-gradient difference. The transferable result is the regime transition at \\kappa=\\sqrt n: using the high-condition-number schedule in the low-condition-number regime wastes oracle calls.",
 "math_tags": [
  "optimization",
  "stochastic-processes",
  "dynamical-systems"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "scheduler"
 ],
 "paper": {
  "arxiv_id": "2609.00045",
  "arxiv_url": "https://arxiv.org/abs/2609.00045",
  "summary_what_math_gives_to_ml": "The paper gives a sharp finite-sum complexity law showing that the useful restart schedule for PAGE depends qualitatively on whether the PL condition number is below or above the square-root sample-size threshold. This is transferable as a condition-number-aware variance-reduced optimizer schedule: use frequent exact refreshes and short inner phases for well-conditioned problems, but retain the longer square-root-scaled PAGE regime for ill-conditioned problems. The dense weak-hiding construction also supplies a principled adversarial benchmark in which individual component gradients reveal little while their exact average preserves the optimization signal.",
  "title": "Dense Weak Hiding: Closing Complexity Gaps in Nonconvex and PL Finite-Sum Optimization under Individual Smoothness",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "speedup",
  "stability"
 ],
 "title": "Condition-number-aware restarted PAGE",
 "url": "https://synthcore.org/idea/2960/condition-number-aware-restarted-page",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}