Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Insert channelwise normalization whose mean and variance are pooled over the full sequence, allowing a small-receptive-field convolutional labeler to access global sequence statistics without adding dilated convolutions or attention. Use this only for tasks where labels occur in long runs or depend on coarse global composition; retain per-position normalization for tasks requiring strict locality.
Treat a neural-network training run as a time-dependent dynamical system and define scalar late-time features that distinguish convergent, oscillatory, noisy, and divergent regimes. Instead of exhaustively sweeping a two-dimensional hyperparameter grid, continue the threshold curve of a feature in the learning-rate/weight-decay or learning-rate/noise plane using a secant predictor and one-dimensional correction sweep. This produces an automatically updated stability map and can be used to keep…
Modify an evolutionary-strategy gradient estimator so that the observed phenotype or trajectory is used to infer the conditional mean of the latent ES perturbation. Instead of multiplying fitness by the raw perturbation, multiply it by the posterior mean perturbation given the realized input; this remains unbiased and has variance no greater than the ordinary ES estimator when the conditional model is correct.
Insert a finite-resolution observation channel between minibatch statistics and the optimizer update, then distinguish information that predicts useful future loss reduction from information that is present in the gradient but has no control value. Use the actionable representation to select the update and suppress increasingly fine, noisy measurements that do not improve progress.
Use the observer contraction rate as an online inference controller. Run the latent observer when its estimated contraction is strong, and invoke expensive retrieval or latent-state reinitialization only when contraction is weak or observation residuals indicate model mismatch.
Maintain a posterior over the effective stochastic-gradient noise scale and trigger expensive diagnostics or conservative optimizer changes only when uncertainty in that scale threatens a training-stability certificate. Unlike entropy-based exploration, the trigger depends on the predicted excess loss or stability gap caused by calibrating the optimizer to the wrong noise level.
Build a low-dimensional neural-network geometry from trainable observables or probes instead of estimating the full Fisher matrix. Precondition the parameter gradient by the inverse variability of the probes and their parameter sensitivity, producing a task-adapted update that can remain usable for implicit models, heavy-tailed data, and parameter-dependent-support distributions.
Use the paper's finite-width O(n^{-1/2}) Gaussian-process approximation bound as a width-budgeting rule rather than choosing every hidden dimension uniformly. Estimate an architecture-specific constant for each layer or attention contraction, then allocate width according to the smallest dimension satisfying its allowed distributional error. This should produce narrower models at comparable GP-like behavior, or permit the same parameter budget to be concentrated in the layers where finite-width…