Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: World model Math: Spectral theory Clear all

✗ Failed on benchmark 2026

Finite-Excitation Latent Replay

Replace derivative-based latent-dynamics fitting with an integral regression and maintain a history stack selected by the smallest eigenvalue of its information matrix. The model should perform aggressive parameter updates only when the estimated latent regressors are sufficiently exciting, while a perturbation bound prevents false excitation caused by inaccurate hidden-state estimates.

Useful8/10

Difficulty5/10

Novelty7/10

Training Stability Sample efficiency Accuracy

Paper: Adaptive Observer of Nonlinear One-Sided Lipschitz Systems Using Estimated State Regressors With Finite Excitation arXiv:2608.30977

△ Mechanism confirmed, baseline not beaten 2026

Greedy Singular-Value Delay Scheduler

Use the observability margin to choose which delay taps to retain under a fixed memory or computation budget. Add a candidate delay only when it substantially increases the smallest singular value of the delay map, converting the paper's large-delay asymptotic result into an adaptive receptive-field construction for sequence models.

Useful7/10

Difficulty5/10

Novelty8/10

Training Stability Memory Accuracy

Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001