Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

✗ Failed on benchmark 2026

Finite-Excitation Latent Replay

Replace derivative-based latent-dynamics fitting with an integral regression and maintain a history stack selected by the smallest eigenvalue of its information matrix. The model should perform aggressive parameter updates only when the estimated latent regressors are sufficiently exciting, while a perturbation bound prevents false excitation caused by inaccurate hidden-state estimates.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Adaptive Observer of Nonlinear One-Sided Lipschitz Systems Using Estimated State Regressors With Finite Excitation arXiv:2608.30977
△ Mechanism confirmed, baseline not beaten 2026

Residual-Christoffel Collocation for Random-Feature PDE Networks

Replace uniform collocation for a fixed random-feature neural PDE solver with sampling from the leverage-score density of the operator-applied features. Whiten the retained residual feature space before solving for output coefficients, so the sampled least-squares matrix has an identity-like expected Gram rather than inheriting severe anisotropy from the differential operator. The same construction can be used for a linearized neural network by treating Jacobian features as the trial functions.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Residual-Christoffel Sampling for Random Feature Collocation of Linear PDEs arXiv:2607.13382
✗ Failed on benchmark 2026

Diffusion-DPP Gradient Batches

Replace uniform minibatch sampling by a fixed-size determinantal point process whose similarity matrix is a diffusion kernel on the training-data k-NN graph. The sampler repels nearby or redundant examples while preserving multiple diffusion modes, so a small batch should cover intrinsic data geometry and provide lower-variance estimates of losses and gradients.

Useful8/10
Difficulty6/10
Novelty5/10
Paper: Fast determinantal sampling on general spaces and diffusion geometry arXiv:2607.06644
✗ Failed on benchmark 2026

Wittrick–Williams Mode Enumerator

Use a spectral eigenvalue-counting function to bracket each target mode before neural optimization. The network then solves only within an interval containing exactly one eigenfrequency, preventing optimization from repeatedly collapsing to the lowest mode or jumping between modes.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response arXiv:2608.28683
△ Mechanism confirmed, baseline not beaten 2026

Greedy Singular-Value Delay Scheduler

Use the observability margin to choose which delay taps to retain under a fixed memory or computation budget. Add a candidate delay only when it substantially increases the smallest singular value of the delay map, converting the paper's large-delay asymptotic result into an adaptive receptive-field construction for sequence models.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001
✓✓ Beats tuned baseline 2026

Laplacian-Coherence Graph Minibatches

Replace uniform node minibatches in a GNN with a coreset selected from a small random candidate set using local Laplacian-column coherence. Select nodes whose connectivity signatures are least redundant with already selected nodes, while retaining inverse-probability weights for unbiased loss estimates. This should improve coverage of weakly connected graph clusters and preserve smooth graph signals at the same batch size.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Scalable Graph Coreset Selection via Greedy Sampling arXiv:2607.27602
△ Mechanism confirmed, baseline not beaten 2026

Adaptive Ballistic-to-Diffusive Propagation Schedule

Use dephasing as a depth- or time-dependent control variable rather than a fixed regularizer: early layers retain coherent transport for feature discrimination, while later layers increase dephasing to eliminate unstable high-frequency oscillations. The schedule is selected from an observable spectral or correlation ratio, giving a falsifiable switch point instead of tuning noise blindly.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Fermions on a 1D lattice: localized sources and sinks with dephasing arXiv:2607.22240
△ Mechanism confirmed, baseline not beaten 2026

Delay-Kernel Bifurcation Scheduler

Use the paper's stability-switching mechanism as a training and inference schedule: begin with a short or broadly distributed delay inside the stable region, then increase the mean delay or concentrate the kernel only when oscillatory or multistable dynamics are useful. The schedule is controlled by the predicted characteristic-root crossing rather than by training step count alone.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Macroscopic Multistability and Bifurcations in Theta-Neuron Networks with Distributed Delays arXiv:2607.17645
✗ Failed on benchmark 2026

Analytic covariance initialization for switched state-space models

Use the paper's finite-dimensional second-moment equations to compute the stationary covariance induced by a Markov-switched recurrent layer before training, then whiten or scale each mode's hidden state using that covariance. This can prevent mode-specific saturation and eliminate a long burn-in period in long-context RNNs and state-space models.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Spectral Characterisation of Covariance Existence in Markov-Switching Affine Recurrences arXiv:2607.09994
Unverified 2026

Nested Bandwidth Feature Curriculum

For bandlimited or progressively higher-frequency targets, construct a nested random Fourier layer whose frequency window grows according to the paper's bandwidth laws instead of sampling all frequencies from one fixed range. Train with a low-bandwidth subset first, then activate additional frequency blocks. This creates a mathematically motivated spectral curriculum and may reduce early optimization difficulty while preserving high-frequency accuracy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral Convergence of Random Feature Method in Multiple Dimensions arXiv:2609.03401
Unverified 2026

Entropy-Adaptive Spectral Groups

Use SPINE's nested entropy profile on the singular values of each trainable weight matrix to discover spectral bands online, rather than choosing a fixed rank or a fixed number of learning-rate groups. Assign smaller step sizes or stronger decay to dominant singular-value bands and larger step sizes to weak bands, while updating the grouping only when the entropy-boundary signal is persistent.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Scale Partitioning by Incremental Nested Entropy: A Measure-Oriented Theory of Multiscale Structure arXiv:2608.17391
Unverified 2026

Chemotactic Instability Spectral Controller

Use the mode-wise instability condition as a controller for a learned cross-channel transport gain. During training or inference, estimate the linearized feature dynamics and adjust the chemotactic strength to remain below a stability margin for robust processing, or deliberately cross the threshold during a controlled pattern-forming stage. This replaces blind gain tuning with a measurable dynamical criterion.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Pattern formation: reactivity is not necessary for chemotaxis--driven instabilities arXiv:2608.03685
Unverified 2026

Hyperuniform Collocation and Minibatch Sampling

Use spatially correlated training points whose low-frequency structure factor vanishes instead of iid points. For neural fields, PINNs, image-coordinate MLPs, or spatially indexed minibatches, this should suppress long-wavelength quadrature and gradient-estimation noise while preserving the represented target dynamics. The finite-order prediction is that a design with structure factor S(k)=O(|k|^{2q}) produces lower variance for smooth losses than iid sampling, especially as the domain or batch…

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Hyperuniform Delone Realizations and Rigidity arXiv:2608.16547
Unverified 2026

Multiplier-Bootstrap Spike Detector

Use multiplier bootstrap on minibatch activation covariances to determine whether a large top eigenvalue is a genuine representation direction or merely a high-dimensional bulk fluctuation. When a spike is repeatedly significant, apply a low-rank whitening or shrinkage correction to that activation subspace; otherwise leave the layer unchanged, avoiding destructive whitening of ordinary bulk variation.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Multiplier Bootstrap and Edge Phase Transitions of High-Dimensional Covariance Matrices arXiv:2608.15053