Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

△ Mechanism confirmed, baseline not beaten 2026

Residual-Christoffel Collocation for Random-Feature PDE Networks

Replace uniform collocation for a fixed random-feature neural PDE solver with sampling from the leverage-score density of the operator-applied features. Whiten the retained residual feature space before solving for output coefficients, so the sampled least-squares matrix has an identity-like expected Gram rather than inheriting severe anisotropy from the differential operator. The same construction can be used for a linearized neural network by treating Jacobian features as the trial functions.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Residual-Christoffel Sampling for Random Feature Collocation of Linear PDEs arXiv:2607.13382
✗ Failed on benchmark 2026

Wittrick–Williams Mode Enumerator

Use a spectral eigenvalue-counting function to bracket each target mode before neural optimization. The network then solves only within an interval containing exactly one eigenfrequency, preventing optimization from repeatedly collapsing to the lowest mode or jumping between modes.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response arXiv:2608.28683
△ Mechanism confirmed, baseline not beaten 2026

Adaptive Ballistic-to-Diffusive Propagation Schedule

Use dephasing as a depth- or time-dependent control variable rather than a fixed regularizer: early layers retain coherent transport for feature discrimination, while later layers increase dephasing to eliminate unstable high-frequency oscillations. The schedule is selected from an observable spectral or correlation ratio, giving a falsifiable switch point instead of tuning noise blindly.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Fermions on a 1D lattice: localized sources and sinks with dephasing arXiv:2607.22240
△ Mechanism confirmed, baseline not beaten 2026

Delay-Kernel Bifurcation Scheduler

Use the paper's stability-switching mechanism as a training and inference schedule: begin with a short or broadly distributed delay inside the stable region, then increase the mean delay or concentrate the kernel only when oscillatory or multistable dynamics are useful. The schedule is controlled by the predicted characteristic-root crossing rather than by training step count alone.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Macroscopic Multistability and Bifurcations in Theta-Neuron Networks with Distributed Delays arXiv:2607.17645
✓✓ Beats tuned baseline 2026

Spectral-Width Prethermal Training Schedule

Use the paper's lifetime law as a controller for training or rollout difficulty. Estimate the active perturbation bandwidth R of hidden states or forecast errors and reduce the residual gain, increase the dispersion order W, or inject controlled bandwidth whenever the estimated prethermal lifetime becomes too short.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: From stable periodic orbits to many-body chaos: doubly tunable prethermalization via engineering of an emergent band structure arXiv:2607.12355
✗ Failed on benchmark 2026

Analytic covariance initialization for switched state-space models

Use the paper's finite-dimensional second-moment equations to compute the stationary covariance induced by a Markov-switched recurrent layer before training, then whiten or scale each mode's hidden state using that covariance. This can prevent mode-specific saturation and eliminate a long burn-in period in long-context RNNs and state-space models.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Spectral Characterisation of Covariance Existence in Markov-Switching Affine Recurrences arXiv:2607.09994
Unverified 2026

Entropy-Adaptive Spectral Groups

Use SPINE's nested entropy profile on the singular values of each trainable weight matrix to discover spectral bands online, rather than choosing a fixed rank or a fixed number of learning-rate groups. Assign smaller step sizes or stronger decay to dominant singular-value bands and larger step sizes to weak bands, while updating the grouping only when the entropy-boundary signal is persistent.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Scale Partitioning by Incremental Nested Entropy: A Measure-Oriented Theory of Multiscale Structure arXiv:2608.17391
Unverified 2026

Chemotactic Instability Spectral Controller

Use the mode-wise instability condition as a controller for a learned cross-channel transport gain. During training or inference, estimate the linearized feature dynamics and adjust the chemotactic strength to remain below a stability margin for robust processing, or deliberately cross the threshold during a controlled pattern-forming stage. This replaces blind gain tuning with a measurable dynamical criterion.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Pattern formation: reactivity is not necessary for chemotaxis--driven instabilities arXiv:2608.03685
Unverified 2026

Hyperuniform Collocation and Minibatch Sampling

Use spatially correlated training points whose low-frequency structure factor vanishes instead of iid points. For neural fields, PINNs, image-coordinate MLPs, or spatially indexed minibatches, this should suppress long-wavelength quadrature and gradient-estimation noise while preserving the represented target dynamics. The finite-order prediction is that a design with structure factor S(k)=O(|k|^{2q}) produces lower variance for smooth losses than iid sampling, especially as the domain or batch…

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Hyperuniform Delone Realizations and Rigidity arXiv:2608.16547
Unverified 2026

Phase-aware Oja preconditioner

Use Oja's streaming eigenvector estimate on a parameter block's incoming gradient stream, but activate its rank-one preconditioning correction only after the mathematically predicted d log d sample threshold. Before that point, the estimate is treated as unreliable and the optimizer remains close to AdamW or SGD. This prevents early noisy spectral directions from destabilizing training while retaining an O(d)-memory alternative to storing a full gradient covariance matrix.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: The Phase Transition in Online PCA Depends on $n/d\log(d)$, not $n/d$ arXiv:2607.23914
Unverified 2026

Dynamical-Degree Expansion Scheduler

Use the tropical dynamical degree as an analytic expansion budget for repeated neural blocks. Layers with $pq>4$ deliberately expand along a known tropical eigendirection, while layers with $pq\leq4$ avoid exponential asymptotic growth; a schedule can therefore increase representational mixing without allowing hidden-state norms to explode.

Useful5/10
Difficulty3/10
Novelty7/10
Paper: Complex dynamics perspective for birational maps of the plane arising from cluster algebra mutations arXiv:2607.08125