Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: Attention Math: Probability Clear all

Unverified 2026

Lambert-W Attention-Range Controller

Use the paper's first-passage crossover as a controller for the communication or attention radius of a multi-agent policy. Instead of fixing a costly global attention range, estimate environmental depletion statistics online and set the radius near the predicted equality of individual and collective search times, expanding it only when the environment enters the collective-search regime. This turns a statistical-mechanical transition formula into an adaptive sparsification rule.

Useful6/10

Difficulty4/10

Novelty8/10

Training Sample efficiency Stability Scalability

Paper: Emergent aggregation from collective foraging arXiv:2608.28046

Unverified 2026

Parity-Copula High-Order Interaction Benchmark

Construct training and evaluation examples whose every (d-1)-variable marginal is exactly independent, but whose full d-variable distribution contains a parity interaction. This isolates genuine high-order reasoning from shortcuts based on pairwise or lower-order statistics and can expose whether attention or MLP architectures learn the intended interaction.

Useful6/10

Difficulty3/10

Novelty6/10

Training Accuracy Generalization Sample efficiency

Paper: How far are $d$-dimensional copulas with uniform $(d-1)$-marginals from (total) independence? arXiv:2608.18286