Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

✗ Mechanism failed 2026

Closure-Decorrelation Memory Scheduler

Choose the neural operator's input-history length from the measured correlation time of the unresolved closure signal produced by coarse-graining. This avoids under-memory, which causes systematic closure error, and over-memory, which increases attention cost and can destabilize training. The same diagnostic can drive adaptive memory truncation across physical regimes.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148
Unverified 2026

Sparse-Cost Temperature Calibration

Use a small set of known or trusted pairwise costs to estimate the effective entropic temperature of a Sinkhorn attention or mixture-of-experts routing layer directly from its observed transport plan. This provides a calibration controller that can detect over-concentrated routing and adjust epsilon without backpropagating through a costly temperature search.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: An exact and fast solution of the inverse Regularized Optimal Transport problem arXiv:2609.01278
Unverified 2026

Parity-Copula High-Order Interaction Benchmark

Construct training and evaluation examples whose every (d-1)-variable marginal is exactly independent, but whose full d-variable distribution contains a parity interaction. This isolates genuine high-order reasoning from shortcuts based on pairwise or lower-order statistics and can expose whether attention or MLP architectures learn the intended interaction.

Useful6/10
Difficulty3/10
Novelty6/10
Paper: How far are $d$-dimensional copulas with uniform $(d-1)$-marginals from (total) independence? arXiv:2608.18286
Unverified 2026

Entropy-Gated Surrogate Slopes

Adapt the slope of each spiking neuron's surrogate derivative using the normalized entropy of its block's attention distribution. High centered entropy uncertainty increases the slope, while low uncertainty decreases it, and a dead zone holds the default slope fixed for ordinary fluctuations. The adaptation exists only in backpropagation, so the forward spike function, parameter count, and inference cost remain unchanged.

Useful5/10
Difficulty4/10
Novelty5/10
Paper: SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers arXiv:2608.13702