ML: Regularization

Machine-learning ideas tagged Regularization in the ML taxonomy of the Math2NN corpus.

Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
Unverified 2026

Equal-amplitude synchronized oscillator modes

Use multiple oscillator modes with weak phase coupling and regularize their active amplitudes toward a common squared amplitude. This transfers the paper's conclusion that coupled nonzero modes satisfy $A_j^2=A_k^2$ or that a mode collapses to zero, producing a controllable mixture of synchronized persistent modes and suppressed modes.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Coupled Van der Pol Networks arXiv:2607.18337
Unverified 2026

Capacity-Triggered Hybrid Optimizer

Replace a continuously tuned optimizer schedule with a three-regime hybrid controller driven by a training-load signal such as an exponential moving average of gradient norm, curvature, loss, or update norm. Below capacity, use the normal optimizer; after a threshold, increase damping or reduce the learning rate; beyond capacity, apply a constrained update such as gradient clipping, step rejection, or gradient accumulation. This imports the paper's finite-capacity and threshold-switching…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Mathematical Model of Dengue Transmission Incorporating Hospital Capacity and Threshold-Based Fogging Interventions arXiv:2607.18140
Unverified 2026

Nested-transport depth consistency

Apply the paper's nested coupling between path distributions at two Krasnosel'skii–Mann depths to an iterative neural block. Penalize discrepancies between intermediate representations using the coupling mass, so that the short unroll learns to approximate the long unroll while preserving the block's actual computational-path geometry. At inference, use the resulting coupled discrepancy as an early-exit criterion.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Krasnosel'skii-Mann iterations beyond asymptotics: a combinatorial analysis arXiv:2607.18121
Unverified 2026

Core-Halo Attention and Routing Regularizer

Add a statistical-complexity maximization term to attention rows or MoE routing distributions so that each probability vector is encouraged to contain a small dominant core and a nearly uniform low-probability halo. Unlike ordinary entropy regularization, this explicitly favors an intermediate concentration regime and predicts a two-level structure: one or a few large probabilities and all remaining probabilities close to one another. The regularizer should use a small coefficient because its…

Useful6/10
Difficulty4/10
Novelty7/10
Paper: A Unified Discrete and Continuous Theory of Core-Halo Complexity Maximizers arXiv:2607.17907
Unverified 2026

Large-Deviation Safety Buffer

Construct a data-dependent threshold for future cumulative safety violations using the paper's decision-centric large-deviation profile instead of a plug-in estimate. For binary violation increments, the threshold explicitly accounts for both uncertainty in the historical violation rate and the probability that the future process produces an unusually large maximum.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Decision-Centric Large Deviations for Data-Driven Capital Buffers in Ruin Models arXiv:2607.17732
Unverified 2026

Barycentric Kirszbraun regularization

Add a sampled multi-point barycentric nonexpansiveness penalty to a neural map instead of enforcing only pairwise Lipschitz bounds. For sampled points and convex weights, penalize output deviation from the corresponding convex combination whenever it exceeds the input deviation. This encourages stable behavior on unseen convex combinations and can constrain a fine-tuned representation to remain geometrically close to a reference map.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Kirszbraun extensions preserving uniform distance in Hilbert spaces arXiv:2607.17672
Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580
Unverified 2026

Paraphrase-Invariant Semantic Posterior

Require the calibrated state posterior to remain unchanged when evidence is presented through information-equivalent prompt templates. Compare state distributions after semantic aggregation rather than raw token probabilities, and add a total-variation consistency penalty during calibration or fine-tuning.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Calibrating Semantic Uncertainty from Observable Language-Model Probabilities arXiv:2607.17447
Unverified 2026

Parity-Constrained Signed Propagation

Replace the unsigned adjacency used by a deep message-passing network with a signing selected from an affine family that makes designated short even cycles unbalanced. Search this family for a small even-power trace, which acts as a proxy for a smaller spectral radius and suppresses explosive long-range propagation. The signing can be fixed before training, so the method adds no per-example inference cost.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Parity families and a kernel-averaged L-function for near-Ramanujan signings arXiv:2607.17343
Unverified 2026

Mean-Scaled Tail Retention Controller

Apply the paper's dynamic truncation rule to per-example gradient norms or activation magnitudes: at each update, retain or downweight only samples whose score is below a threshold proportional to the current mean score, while explicitly compensating for the resulting selection bias. This creates a controllable tail-removal process whose fixed point and sensitivity to score variance can be measured before committing to large experiments.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Adverse Selection with Quality Variance: A Maximum-Entropy Approach arXiv:2607.17239
Unverified 2026

Margin-stabilized stateful routing

Add a continuation-value correction and an explicit positive margin to routing or adaptive-computation decisions that consume a stateful resource. The margin prevents the degeneracy in which learned rent exactly cancels immediate surplus, making the network highly sensitive to noise, quantization, or small state-estimation errors.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Certified-Gap Dual-Price Policies for Real-Time Truckload Bid Acceptance with Relocating, Clock-Constrained Resources arXiv:2607.16891
Unverified 2026

Rigid-Motion-Quotient Covariance Loss

Add a distribution-level loss that compares minibatch embeddings only through the square roots of their ordered covariance eigenvalues, ignoring global translation and rotation of the embedding coordinate system. This implements the Gaussian specialization of the paper’s Procrustes-Wasserstein geometry and is useful when two embedding clouds are semantically equivalent up to a rigid change of coordinates.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Dynamical Optimal Transport with $\mathfrak{so}(d)$-Invariance: From Theory to Computation arXiv:2607.16782
Unverified 2026

Backward-Only Residual Masking

Replace forward activation dropout in a deep residual network with an unbiased multiplicative mask applied only to gradients entering each residual unit or branch. The forward representation remains deterministic for a given minibatch, while the update receives zero-mean stochastic perturbations that are predicted to reproduce dropout-like limiting dynamics in large-width, large-depth ResNets.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNets arXiv:2607.16761
Unverified 2026

All-Cut Schatten Control for Polynomial Layers

Replace ordinary Frobenius or spectral-norm control of a tensorized multilinear layer by a sampled approximation to its oriented Schatten profile, the maximum Schatten norm of every input-output flattening. Regularizing this profile should control Gaussian or randomized polynomial activations uniformly over hidden width and tensor contraction pattern, reducing exploding activations and making higher-order layers easier to scale.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Algebraic Transfer for Operator-Valued Gaussian Chaoses:Oriented Schatten Profiles and Singular Wick Multipliers arXiv:2607.16724
Unverified 2026

Hysteretic Continuation Controller

Use the external field H as a slowly swept control variable for a neural module, loss coefficient, or optimizer gain, and deliberately retain the resulting branch memory instead of replacing it with an instantaneous equilibrium update. Forward and backward sweeps produce a hysteresis loop whose shape diagnoses first-order-like training transitions, while controlled disorder changes the loop area and can suppress harmful branch dependence. This supplies a continuation-based curriculum or…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Glauber dynamics phase transitions in athermal random field Blume-Capel and Blume-Emery-Grifitths models arXiv:2607.16561
Unverified 2026

C1 Homogeneous Lyapunov Critic

For a neural ODE or recurrent state update, learn a positive-definite degree-two homogeneous Lyapunov function that is only C1, rather than restricting the certificate to polynomials or analytic neural networks. Parameterize its angular dependence with a positive spline or softplus mixture, and train it to decrease along the learned vector field; this can certify stable dynamics that polynomial Lyapunov searches systematically miss.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: A Globally Asymptotically Stable Planar Homogeneous Polynomial Vector Field With No Polynomial Lyapunov Function arXiv:2607.16171
Unverified 2026

Laplace-Margin Regularized Depression RNN

Augment a recurrent or state-space layer with a bounded synaptic-depression variable that multiplicatively reduces recurrent transmission after activity. During training, estimate the layer's impulse-response transform and penalize characteristic roots approaching the unstable half-plane. This directly targets slow oscillations and exploding recurrent feedback rather than relying only on gradient clipping.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On large networks of integrate-and-fire neurons with short-term synaptic plasticity arXiv:2607.16017
Unverified 2026

Scale-Slack Homology Consistency

Regularize a neural embedding so that two augmented views of the same point cloud or graph induce homologous cycles whenever their embedded vertices move by at most δ. Instead of requiring identical topology at exactly the same distance threshold, compare homology at ε for one view with homology at ε+δ for the other, matching the paper's mathematically justified scale slack. This should discourage brittle holes and connected-component changes caused purely by augmentation noise while…

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Skeletal Homology arXiv:2607.16009
Unverified 2026

Minkowski gradient barrier for neural fields

Add a Born–Infeld/Minkowski-gradient barrier to a coordinate MLP so that its spatial gradient remains below a prescribed speed limit, rather than using an ordinary quadratic smoothness penalty. Couple the barrier with a forcing or task loss; under strong forcing, the resulting field should preferentially approach a distance-to-boundary-like profile while avoiding exploding derivatives and oscillatory solutions.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Positive and nodal solutions for the Minkowski mean curvature equation: multiplicity and asymptotics arXiv:2607.15956
Unverified 2026

Barrier-Ultrametric Trust Regions

Construct a barrier metric between neural-network checkpoints or low-loss states using transition rates on a sparse neighbor graph, and use its induced single-linkage hierarchy to restrict updates within the current basin before permitting cross-basin moves. In the large barrier-spread regime, the metric is controlled by the largest barrier along the best path, producing an ultrametric hierarchy that can replace unreliable Euclidean distance for trust-region and replay decisions.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Ultrametric organization of energy landscapes on random Erdős--Rényi graphs: topological origin of barrier hierarchy arXiv:2607.15902
Unverified 2026

Fourier-compressed attention bias

Replace an unconstrained relative-position attention-bias table by a learnable two-dimensional Fourier representation, or regularize the bias toward a small Fourier ratio. The ratio favors coefficient concentration without depending on the absolute scale of the bias, allowing the trained bias to be reconstructed from a small number of dominant frequencies.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Edge complexity of graphs arXiv:2607.15598
Unverified 2026

Two-budget Newton continuation for max-affine adapters

Represent a small vector of neural control variables as a two-objective max-affine feasibility problem, such as clean-loss budget versus corruption-loss budget or task-performance budget versus activation-range budget. Rather than launching many independent weighted-sum optimizations, construct the active linear boundary and jump from breakpoint to breakpoint with the paper's Newton continuation step. This is intended for frozen-backbone calibration, LoRA gain tuning, activation clipping, or…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Tropical Bi-Objective Pseudolinear Optimization as Parametric Mean-Payoff Games arXiv:2607.15481
Unverified 2026

Calibrated Prediction-Mixed Distillation

Use fresh unlabeled covariates to train a frozen-teacher student against pseudo-labels, then form an affine combination of teacher and student predictions. Estimate the combination weight on a small independent labeled calibration set, requiring no access to the teacher training data and no additional teacher or student fitting.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Prediction-Only Distillation in Linear and Logistic Regression arXiv:2607.15450