Curvature-Guided Discrepancy Gradient Accumulation
Implementation & benchmark of arXiv:2607.08943 — Online Komlós converges to mean curvature flow
Source paper: Online Komlós converges to mean curvature flow arXiv:2607.08943 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use sign choices over redundant gradient or adapter proposals to keep the accumulated residual update small in the coordinatewise maximum norm. Constrain the sign controller to preserve a positive projection onto the desired descent direction, so it suppresses coordinate spikes without completely canceling optimization progress.
Formulas
Mathematical statement
The paper studies a state y in R^m initialized at y_0 = 0. At round t, an adversary supplies n vectors a_{t,j} in the Euclidean unit ball, and the controller chooses signs epsilon_{t,j} in {-1,1}; the state update is y_t = y_{t-1} + sum_j epsilon_{t,j} a_{t,j}. The controller minimizes the final infinity norm ||y_T||_infinity = max_i |(y_T)_i|. The paper states that the large-T game value has leading term sqrt(T/(2 tau)), where tau is the extinction time of the unit cube in R^m under a curvature-based flow. When n is at least m - 1, the flow is mean curvature flow and 1/sqrt(2 tau) = Theta(sqrt(log m)). For the neural adaptation, g_t is the desired normalized gradient, r_{t,j} are redundant candidate residual updates, b_t is the accumulated residual state, and epsilon_{t,j} are selected signs. We add the engineering constraint that the signed update has positive descent projection, namely inner product(g_t, sum_j epsilon_{t,j} r_{t,j}) >= rho ||g_t||_2^2 for rho > 0. This descent constraint is an adaptation, not part of the paper's game theorem. The transferable mathematical asset is the infinity-norm discrepancy objective and its predicted O(sqrt(T log m)) coordinatewise growth under bounded vectors and sufficiently many sign-controlled proposals.
Implementation notes
Integrate the method into gradient accumulation before the optimizer update. Generate n redundant candidate updates per training step by computing gradients on n microbatches, using n dropout masks, or producing n independent low-rank adapter proposals. Flatten the selected parameter block into m coordinates and denote the candidates by r[t,j]. Normalize each candidate so its Euclidean norm is at most one, while storing the removed scale separately. Let g[t] be the ordinary minibatch gradient or the mean candidate gradient. Maintain an accumulated residual b. Select signs using either exhaustive enumeration when n is at most 8 or a greedy coordinate-descent search for larger n: initialize all signs to plus one, then visit candidates and flip sign j if the flip reduces the infinity norm of b plus the signed candidate sum and preserves the descent condition inner_product(g[t], signed_sum) >= rho times squared_norm(g[t]). If no flip is feasible, retain the feasible sign pattern with the smallest infinity norm. Add the selected signed sum to b. Every K rounds, pass b/K to SGD or AdamW as the update and reset b to zero. The paper-derived quantities are the infinity-norm discrepancy objective and the predicted square-root growth proportional to sqrt(T log m); the descent projection, greedy solver, finite accumulation horizon K, and candidate-generation mechanism are empirical adaptations. Estimate the scaling exponent by plotting median infinity norm of b against accumulation length on log-log axes. Start with a 2-layer MLP or ResNet-18 on CIFAR-10, using n equal to 4 or 8 microbatches. Compare ordinary accumulation, random signs, and greedy discrepancy signs at equal examples and FLOPs. Measure peak coordinate magnitude, gradient-clipping frequency, loss variance, final accuracy, and optimizer-state memory. A successful result is at least a 2x lower residual infinity norm and fewer clipping events at equal validation accuracy, with no more than 1% relative degradation in training loss.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built the sign-controller MVP and verified the core discrepancy phenomenon numerically: guided signs reduced the median infinity-norm residual from 14.1–32.2 (all-plus) and 15.3–29.1 (random) to 0.83–3.19 across m=4–64, with approximately zero temporal growth exponent versus about 0.48–0.51 for controls. However, in the fixed-seed MLP test the guided method reduced peak residual median from 1.015 to 0.186 but degraded accuracy from 91.9% to 51.8% and increased loss from 0.510 to 0.683, so the proposed adaptation did not produce a useful optimization win.
- Agent confidence
- 9/10
- Baseline
- Toy all-plus: median ||b||inf 14.10, 22.06, 22.19, 26.58, 32.16 for m=4,8,16,32,64; random: 15.25, 18.89, 23.87, 25.58, 29.12. MLP: peak residual median 1.015, max 2.685, accuracy 0.919, loss 0.510.
- Idea
- Toy guided: median ||b||inf 0.981, 0.832, 1.669, 1.985, 3.190 for m=4,8,16,32,64; m=32 growth exponent -0.0002 versus 0.482 all-plus and 0.514 random. MLP: peak residual median 0.186, max 0.498, accuracy 0.518, loss 0.683.
Limitations: Only a tiny synthetic 2D classification task and one fixed seed were tested; no CIFAR-10, dropout/microbatch redundancy alternatives, AdamW, FLOP-normalized comparison, multiple seeds, GPU path, clipping-frequency difference, or optimizer-state memory measurement was evaluated. The toy controller used exact enumeration for n=4 and therefore does not validate large-n greedy-search quality.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- discrepancy.py 2.3 KB View Raw
- experiment.py 3.9 KB View Raw
- report.md 1.7 KB View
- results.json 1.4 KB View Raw