Math: Singular perturbation

Machine-learning ideas tagged Singular perturbation in the Math taxonomy of the Math2NN corpus.

2 ideas found

Status: Unverified Math: Singular perturbation Clear all

Unverified 2026

Diffusive fast-slow recurrent block

Replace a standard recurrent update with a slow-fast oscillator whose fast hidden state is coupled across feature channels by a graph-Laplacian diffusion term. The slow-fast structure permits sharp transient transitions, while diffusion suppresses unstable disagreement modes and should make long unrolled computation less sensitive to initialization and perturbations.

Useful6/10

Difficulty5/10

Novelty6/10

Dynamics Stability Accuracy Generalization

Paper: Diffusion stabilises time-periodic solutions in conservation laws coupled to a relaxation oscillator arXiv:2607.24994

Unverified 2026

Singularly Perturbed Hierarchical Training

Train the output layer on a fast timescale and the hidden feature layer on a slow timescale, so output coefficients first fit the components representable by the current features before hidden directions move. Use residual plateaus to detect when the fast subsystem has approximately equilibrated, then increase the hidden-layer learning rate to begin the next feature-learning stage.

Useful6/10

Difficulty4/10

Novelty5/10

Dynamics Stability Sample efficiency Accuracy

Paper: Singular perturbations and hierarchical learning in two-layer neural networks arXiv:2607.10869