Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

△ Mechanism confirmed, baseline not beaten 2026

Matrix-Free Krylov Backpropagation Through Solver Layers

Turn an iterative optimization or equilibrium computation inside a neural network into a differentiable layer whose backward pass solves the implicit adjoint system with conjugate gradients or GMRES using only automatic-differentiation matrix-vector products. This avoids storing unrolled iterations and avoids explicit Hessian or Jacobian construction, enabling longer solver horizons and lower-memory implicit architectures.

Useful8/10
Difficulty6/10
Novelty5/10
Paper: PANDA: A Matrix-Free Differentiable NMPC Solver via Proximal Averaged Quasi-Newton with Adaptive Linesearch Algorithm arXiv:2608.16280
✗ Mechanism failed 2026

Conformal Early-Rejection for Diffusion Architecture Search

Attach a calibrated risk monitor to intermediate diffusion states and terminate mutations that are likely to violate hard architecture or performance constraints before full decoding and training. This transfers the paper's separation between proposal generation and authoritative external evaluation into an early-stopping controller for expensive neural architecture trials.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: From Generation to Discovery: Diffusion Mutation Kernels for Circuit and Physical Design arXiv:2608.27649
△ Mechanism confirmed, baseline not beaten 2026

Self-Supervised Amortized Mean-Field Controller

Train one prompt-conditioned controller to solve a distribution of stochastic control tasks directly from the control objective, instead of generating an optimal trajectory dataset for every task. Use the probability-flow velocity to evolve particles deterministically, evaluate running and terminal costs on those particles, and backpropagate through the rollout to learn a reusable operator.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Self-supervised In-context Operator Learning for Stochastic Mean-Field Control arXiv:2608.18282
△ Mechanism confirmed, baseline not beaten 2026

Branch-Free Double-Word FMA Accumulator

Replace ordinary low-precision multiply-add accumulation in selected neural-network reductions with a two-word floating-point accumulator updated by the paper's branch-free DW-FMA network. The high word retains the main sum and the low word stores the rounding residual, improving cancellation behavior without the control-flow divergence of conditional compensated summation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Performance evaluation of branch-free fused multiply-add algorithms for multi-component-type multiple-precision floating-point arithmetic arXiv:2607.11391
Unverified 2026

Stochastic Operator Stability Monitor

Instrument selected neural-network operators with cheap stochastic perturbations and estimate how much their outputs change under finite-precision perturbations. Use the resulting per-operator score to identify unstable kernels and selectively switch them to FP32, compensated accumulation, or a stable reformulation instead of running the entire model at high precision.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Automated Numerical Stability Analysis of Deep Learning Operators arXiv:2607.25494
Unverified 2026

Dynamical-Degree Expansion Scheduler

Use the tropical dynamical degree as an analytic expansion budget for repeated neural blocks. Layers with $pq>4$ deliberately expand along a known tropical eigendirection, while layers with $pq\leq4$ avoid exponential asymptotic growth; a schedule can therefore increase representational mixing without allowing hidden-state norms to explode.

Useful5/10
Difficulty3/10
Novelty7/10
Paper: Complex dynamics perspective for birational maps of the plane arising from cluster algebra mutations arXiv:2607.08125