Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: Transformer Math: Stochastic processes Clear all

✗ Mechanism failed 2026

Conformal Early-Rejection for Diffusion Architecture Search

Attach a calibrated risk monitor to intermediate diffusion states and terminate mutations that are likely to violate hard architecture or performance constraints before full decoding and training. This transfers the paper's separation between proposal generation and authoritative external evaluation into an early-stopping controller for expensive neural architecture trials.

Useful7/10

Difficulty5/10

Novelty7/10

Training Speedup Sample efficiency Scalability

Paper: From Generation to Discovery: Diffusion Mutation Kernels for Circuit and Physical Design arXiv:2608.27649

△ Mechanism confirmed, baseline not beaten 2026

Lyapunov-Calibrated Multiplicative Noise

Use measured local Jacobian growth to set the variance of dropout, feature noise, or stochastic-depth perturbations, implementing the paper's fluctuation-response idea that multiplicative noise is tied to the positive scrambling or Lyapunov rate. The controller maintains a target growth regime instead of applying a fixed noise schedule throughout training. It predicts a stability transition when the estimated growth rate crosses zero and a variance-growth proportionality that can be tested…

Useful7/10

Difficulty4/10

Novelty6/10

Training Stability Generalization Scalability

Paper: Effective Field Theory of Operator Scrambling from Strong-to-Weak Symmetry Breaking arXiv:2607.24925