Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: Attention Math: Numerical analysis Clear all

✗ Mechanism failed 2026

Closure-Decorrelation Memory Scheduler

Choose the neural operator's input-history length from the measured correlation time of the unresolved closure signal produced by coarse-graining. This avoids under-memory, which causes systematic closure error, and over-memory, which increases attention cost and can destabilize training. The same diagnostic can drive adaptive memory truncation across physical regimes.

Useful7/10

Difficulty4/10

Novelty7/10

Training Accuracy Stability Speedup

Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148

△ Mechanism confirmed, baseline not beaten 2026

Branch-Free Double-Word FMA Accumulator

Replace ordinary low-precision multiply-add accumulation in selected neural-network reductions with a two-word floating-point accumulator updated by the paper's branch-free DW-FMA network. The high word retains the main sum and the low word stores the rounding residual, improving cancellation behavior without the control-flow divergence of conditional compensated summation.

Useful7/10

Difficulty5/10

Novelty6/10

Training Accuracy Stability Speedup

Paper: Performance evaluation of branch-free fused multiply-add algorithms for multi-component-type multiple-precision floating-point arithmetic arXiv:2607.11391