Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: Regularization Math: Geometry Clear all

✗ Mechanism failed 2026

Robustness-capacity feasibility controller

Use the paper's lower bound as a feasibility test for robust interpolation: if a model is asked to fit below the estimated noise floor while maintaining a small Lipschitz constant, automatically increase effective width or relax the fit target. This prevents optimization from wasting compute on an impossible low-sensitivity solution and provides a principled width schedule for noisy regression or classification.

Useful6/10

Difficulty4/10

Novelty7/10

Training Stability Generalization Accuracy

Paper: A law of robustness for two-layer neural networks with arbitrary weights arXiv:2607.07778

△ Mechanism confirmed, baseline not beaten 2026

Convex-gradient robust augmenter

Replace unconstrained adversarial example generation with an invertible transport map that is the gradient of a convex potential. For each class, the map pushes a kernel-smoothed empirical distribution toward a least-favorable distribution inside a prescribed KL/Sinkhorn ambiguity radius, producing hard but globally coherent training examples rather than pointwise perturbations.

Useful6/10

Difficulty6/10

Novelty5/10

Training Accuracy Generalization Stability

Paper: Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing arXiv:2608.22746