Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training Solves: Speedup ML: Regularization Clear all

✗ Failed on benchmark 2026

Barrier-Controlled Basin Switching

Use a learned quasipotential barrier as feedback for optimizer noise and restart control. Increase stochasticity when training is trapped in a high-loss metastable basin and reduce it near a desirable basin, with switching thresholds determined by the estimated barrier rather than by a fixed patience schedule.

Useful7/10

Difficulty6/10

Novelty7/10

Training Stability Speedup Accuracy

Paper: Stochastic Dynamics of the Two-Dimensional Low-to-High Transition System Driven by Multiplicative Noise arXiv:2607.23186

Unverified 2026

Criticality-controlled sparsity schedule

Use the min-plus phase transition as a training-time controller: begin near p = 1/2 to preserve the initial active-state fraction across depth, then move above or below criticality to deliberately remove or create sparse pathways. The controller uses a measurable state variable, the activation zero fraction, rather than an arbitrary regularization coefficient.

Useful6/10

Difficulty5/10

Novelty9/10

Training Stability Regularization Speedup

Paper: Finite-depth scaling and an exact Bernoulli-leaf identity for the min-plus process on the binary tree arXiv:2608.12295