Machine-learning ideas tagged Regularization in the ML taxonomy of the Math2NN corpus.
Use the logarithmic transmission amplitude of an SU(1,1) scan as a differentiable spectral penalty. The paper's constant-one nonlinear Hausdorff–Young inequality provides a principled upper budget for this amplitude in terms of the input L^p norm, replacing an arbitrary spectral-weight penalty with a scale-aware constraint.
Replace a conventional covariance or density-matrix discrepancy with the geodesic quantum f-divergence between an example's predicted positive-definite matrix and its target matrix. Use t as a controllable interpolation between the standard Petz divergence at t=0 and the maximal divergence at t=1, with f(x)=x log x or another operator-convex power generator. The loss is suited to covariance-predicting networks, SPD-valued embeddings, and matrix-valued classifiers where eigenvector alignment…
Use adaptive performance specifications to prevent a neural controller or policy from demanding output changes that exceed bounded actuator amplitude or action-rate limits. The target error envelope tightens when the policy has control authority and relaxes when saturation or rate clipping persists, instead of allowing the controller to destabilize while chasing an infeasible target. This converts actuator clipping into an explicit slow state that can be used by reinforcement-learning policies…
Replace ordinary Frobenius-norm clipping when merging rank-one LoRA or adapter updates with a Schatten-budget computed from the positive operators |A_k|. For p>=2, the paper's sharp rank-one inequality bounds the norm of the merged update, including interactions between updates that are missed by independent per-update clipping.
Use generalized dual numbers to compute second- or third-order derivatives of the training loss along several parameter-space directions, then use polarization to recover mixed directional derivatives without forming a Hessian or third-order tensor. Add a bounded mixed-curvature penalty or use the resulting directional curvature to rescale updates in directions that are simultaneously sharp.
Add a fractional Sobolev penalty to the spatial output of a neural field or reconstruction CNN, rather than relying only on pixelwise weight decay or total variation. The fractional order s continuously controls high-frequency suppression, allowing an experiment to test whether s less than 1 preserves edges better than the classical integer-order penalty while still reducing noise and unstable oscillations.
Add a mean-preserving periodic-input consistency penalty to a stacked leaky recurrent or state-space network. The penalty suppresses output shifts caused purely by hidden-state fluctuations and nonlinear curvature, improving invariance to temporal modulation while preserving the average input signal.
Replace a standard graph-convolution propagation step with a short time integration of the nonlinear graph flow \(\partial_t u=\Delta_p(u^q)\). The pointwise power \(q\) and gradient exponent \(p\) create state- and edge-gradient-dependent propagation: small signals can be suppressed or amplified by \(q\), while large graph discrepancies receive nonlinear diffusion controlled by \(p\). Use nonnegative feature states and conservative edge fluxes so the layer inherits positivity and total-mass…
Add a multiscale penalty to transformer token-mixing activations when they are simultaneously concentrated on a spatial or token subset and on a separated, irregular frequency subset. The penalty uses the fractal uncertainty scaling law to discourage hidden states from collapsing onto narrow token patterns and narrow spectral bands, potentially improving robustness to token masking and frequency-corrupted inputs.
Replace fixed-path robustness testing with a coupled continuation procedure that increases an adverse perturbation while simultaneously optimizing a bounded corrective response, such as feature-gating, normalization, or a small adapter. Define the model's margin as the cumulative perturbation at which its equilibrium, prediction, or input-output Jacobian becomes singular or exceeds a prescribed gain threshold; train the corrective response to enlarge this margin subject to an explicit cost.
Constrain the Jacobian of a recurrent or state-space transition to preserve a prescribed cone of admissible hidden-state perturbations. This imports differential positivity into neural dynamics and makes long-run hidden trajectories order-preserving rather than allowing arbitrary sign-changing perturbation growth.
Insert a fixed or learnable multiresolution transform before a CNN or vision-transformer block and penalize its coefficients with the paper's local tent-space square function. The penalty couples coefficients belonging to the same spatial dyadic region and can remove localized multiscale feature packets, potentially producing structured sparsity and better denoising than independent l_1 shrinkage.
Replace raw squared penalties on generated feature means with the paper's nested information-projection statistic. A model output distribution is projected once onto structural constraints and once onto structural-plus-test constraints; their KL divergence produces a sample-size-scaled loss and an approximate chi-square p-value. This should help when constraints have different variances or are strongly correlated, because the KL geometry automatically adapts to their covariance instead of…
For neural eigenmode solvers on periodic domains, train the full field directly and impose Bloch phase coupling only at opposite cell boundaries, rather than differentiating a periodic factor with respect to q through a quadratic volume operator. The boundary formulation preserves reciprocal-lattice equivalence exactly through z=exp(iqa), reducing spurious eigenmodes caused by inconsistent q-dependent discretization.
Given a learned recurrent dynamics map, estimate a state-dependent invariant measure from each trajectory and use integration against that measure as a projection onto long-term invariant features. Penalize discontinuities of this projection between nearby states and assign zero mass to trajectories whose feature norms escape, producing a principled distinction between convergent attractors and divergent rollouts.
Replace a fixed KL or Jensen-Shannon penalty with a learnable Csiszár f-divergence whose generator is parameterized so that convexity is guaranteed. Apply it between teacher and student distributions, augmentation views, or intermediate representations; the loss cannot increase after a stochastic channel such as augmentation, pooling, token merging, or quantization, making the regularizer structurally compatible with information-discarding network operations.
Separate low-support monomials, which involve only a few distinct input coordinates, from high-support monomials in a high-degree symmetric interaction layer. Compute the low-support orbit features exactly and prune, sample, or factorize the high-support tail, using the paper's cutoff scale as the initial sparsity rule.
Replace an unconstrained Wasserstein representation-matching loss with a graph-causal transport loss whose coupling at node k is conditioned only on the representations of its parents. This forces domain alignment, distillation, or augmentation consistency to respect the information flow of the model's DAG, reducing spurious matches that exploit descendants or globally visible features.
Regularize a neural encoder so its local pullback metric is bounded by the refined Schwarz-lemma constant instead of using a generic Frobenius Jacobian penalty. For an encoder into a negatively curved latent space, penalize only singular directions whose squared expansion exceeds the curvature- and dilatation-dependent threshold.
Represent recurrent hidden states as compact phases and monitor spacetime vortices, defined by wrapped phase differences around elementary space-time plaquettes. Add a feedback controller that increases relaxation toward the homogeneous phase when vortex activity becomes supercritical, while allowing larger recurrent gain when the system is excessively quiescent. This creates a falsifiable operating regime: useful computation should occur near, but below, the defect-proliferation transition…
Learn an endpoint-conditioned scalar potential whose level sets represent states with the same asymptotic behavior, analogous to the paper's stable magnetic orthospheres. Train the dynamics to contract differences within a level set while preserving differences between distinct endpoint classes, producing a latent representation organized by stable manifolds rather than Euclidean proximity.
Replace independent additive noise on spatial feature maps with stochastic advection by divergence-free vector fields. The perturbation preserves spatial volume and feature mass, while the associated Stratonovich-to-Itô correction provides a tunable diffusion that preferentially damps high-frequency spatial fluctuations.
Replace an unconstrained deep residual recurrence by a discretized diffusion system over feature or token positions, with trainable source terms and analytically constrained boundary feedback. The state remains nonnegative under nonnegative inputs, while negative boundary gains enforce exponential decay of perturbations and prevent exploding activations in very deep stacks.
Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…