Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training Solves: Memory ML: Regularization Clear all

△ Mechanism confirmed, baseline not beaten 2026

Greedy Singular-Value Delay Scheduler

Use the observability margin to choose which delay taps to retain under a fixed memory or computation budget. Add a candidate delay only when it substantially increases the smallest singular value of the delay map, converting the paper's large-delay asymptotic result into an adaptive receptive-field construction for sequence models.

Useful7/10

Difficulty5/10

Novelty8/10

Training Stability Memory Accuracy

Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001

Unverified 2026

p-Consensus Synchronization for Federated Replicas

Add asynchronous nonlinear consensus steps between local-SGD updates in a federated or decentralized system. Instead of averaging a participating client with all peers, replace one client parameter vector by the coordinatewise p-mean of neighboring replicas, allowing sparse communication and reduced sensitivity to atypical client models.

Useful5/10

Difficulty5/10

Novelty7/10

Training Stability Memory Generalization

Paper: Consensus time for asynchronous $\ell^p$ relaxation: graph dependence arXiv:2609.03856