Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

△ Mechanism confirmed, baseline not beaten 2026

Branch-Free Double-Word FMA Accumulator

Replace ordinary low-precision multiply-add accumulation in selected neural-network reductions with a two-word floating-point accumulator updated by the paper's branch-free DW-FMA network. The high word retains the main sum and the low word stores the rounding residual, improving cancellation behavior without the control-flow divergence of conditional compensated summation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Performance evaluation of branch-free fused multiply-add algorithms for multi-component-type multiple-precision floating-point arithmetic arXiv:2607.11391
Unverified 2026

Sparse-Cost Temperature Calibration

Use a small set of known or trusted pairwise costs to estimate the effective entropic temperature of a Sinkhorn attention or mixture-of-experts routing layer directly from its observed transport plan. This provides a calibration controller that can detect over-concentrated routing and adjust epsilon without backpropagating through a costly temperature search.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: An exact and fast solution of the inverse Regularized Optimal Transport problem arXiv:2609.01278
Unverified 2026

Order-Parameter Mode Activation Schedule

Use the paper's order-parameter dynamics to initialize spectral feature modes with deliberately separated activation times. This creates a controlled progressive-learning curriculum in which dominant modes become available first and weaker modes activate later, potentially reducing early gradient interference.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws arXiv:2608.13335