Machine-learning ideas tagged Stability in the Solves taxonomy of the Math2NN corpus.
Partition activations into dyadic magnitude bands and allocate sparse connectivity separately to heavy and diffuse coordinates. Protect high-magnitude coordinates with more reliable connections while using randomized flat connectivity for the many small coordinates, keeping the total number of nonzeros fixed.
Replace a one-step recurrent update with a causal convolution over past affine innovations using an exponential-fractional kernel. Add mean reversion and calibrate the innovation amplitude so that activation mean and variance remain approximately invariant across sequence position while retaining long-range, power-law-like memory.
Replace one local spatial aggregation in a CNN or vision transformer with a discretized Riesz potential whose kernel is proportional to $\|x-y\|^{-(n-s)}$. Normalize the layer using the paper's sharp weak-type constant and penalize empirical violations of the resulting tail bound, encouraging nonlocal context without allowing a small set of pixels or tokens to generate arbitrarily large responses.
Use the paper's analytic merging threshold to choose the Scaled-NAP exponent from an intended community size rather than treating alpha as an arbitrary hyperparameter. A warm-started schedule can begin with persistence-like fine structure and increase alpha only when the model has learned reliable local groups.
Use a Goldfarb–Idnani-style active-set solver as a neural constrained layer or optimizer substep, but never trust a guessed active set solely because its linear system solved. Remove duplicate or dependent constraints, solve the reduced KKT system, and accept the result only after checking primal feasibility, dual sign conditions, and stationarity. This gives exact enforcement of linear inequalities and a diagnostic certificate when the constraint set is infeasible.
Replace ordinary per-coordinate activation normalization or covariance whitening with a learned or frozen affine map based on a minimum-volume ellipsoid calibrated to the activation convex hull. Estimate the hull's Minkowski symmetry and use the bound \(\rho=\sqrt{d/\alpha}\) to set a principled clipping radius, where \(d\) is feature dimension and \(\alpha\) is the measured symmetry. This should reduce worst-case clipping of asymmetric activation distributions without requiring a much larger…
Add a differentiable penalty that encourages a neural implicit field to have a controlled local homogeneity degree across concentric spatial scales. The penalty compares the flux-normalized frequency at adjacent radii, optionally targeting a desired degree k, so the network is discouraged from producing scale-inconsistent or oscillatory local geometry.
Replace a channelwise wavelet or strided-convolution front end with vector-valued wavelet filters that deliberately pair different scalar wavelets across channels. The resulting subbands retain compact-support multiscale structure and can be recombined exactly, while a small learned 1x1 mixing layer operates on the cross-channel coefficients instead of learning a full expensive convolution at every scale.
Replace two unconstrained scalar quadratic feature heads with a pair whose quadratic forms admit a positive-definite linear combination. This prevents the two heads from simultaneously vanishing on any nonzero hidden vector, which can reduce representation collapse and improve the conditioning of downstream gates or auxiliary objectives. The constraint can be implemented softly with a spectral-margin penalty, or exactly by parameterizing one learned pencil as positive definite.
Replace raw powers or unconstrained polynomial spectral features with normalized Jacobi features whose amplitude is provably bounded on the entire input interval. Use trainable mixtures of these features in a positional encoding, graph spectral layer, or MLP front end, while preserving the theorem's normalization and optionally constraining the learned mixture norm.
Replace a hard nonnegative slack or ReLU output by the barrier-derived map \(x_s(w)=\frac{w+\sqrt{w^2+4s}}{2}\). Unlike an arbitrary smooth activation, this output is the unique positive solution of \(x(x-w)=s\), so the network can explicitly monitor complementarity and anneal \(s\) toward the true inequality-constrained solution. Use it in a constrained output head or in hidden layers whose activations represent nonnegative resource, probability, or routing slack variables.
Use the adjacency matrix of a vertex-transitive strongly regular graph as a fixed sparse attention or token-mixing mask. Every vertex has the same degree, and every pair of vertices has exactly one of two common-neighbor counts, giving predictable two-hop coverage and avoiding the degree and connectivity irregularities of random sparsification.
Replace raw polynomial features in a scalar MLP expansion with endpoint-weighted orthonormal Jacobi features. The paper's envelope gives a degree- and parameter-aware scale for each feature, preventing high-degree terms or endpoint behavior from dominating gradients while preserving a richer approximation basis than low-degree monomials.
Replace an unconstrained low-rank adapter or similarity projection with a learned subspace carrying a prescribed signed metric. The module learns an orthonormal basis U for a k=p+q dimensional subspace, forces the compressed form U^*I_{m,n}U to have p positive and q negative eigenvalues, and uses the resulting pseudo-inner product for signed attention or retrieval scores.
Use the Gaussian approximation of a high-dimensional maximum to set a simultaneous coordinate-clipping threshold for minibatch gradients or activations. The threshold is sampled from a correlated Gaussian with the observed batch covariance, rather than treating coordinates as independent or estimating an unstable extreme quantile directly.
Replace random or learned routing hashes for tokens arranged on a d by d grid with a fixed family of mutually orthogonal anti-Latin squares. Each channel assigns exactly d of the d squared tokens to every bucket, while any two channels jointly distinguish every grid position. The resulting router has deterministic load balance and multi-view positional diversity.
Replace a globally shared latent transformation group by a source-dependent collection of valid transformation paths. A feature at latent point z is transported only along paths whose transformed coordinate never reaches the singular locus, while homotopic paths are identified and composable paths are concatenated. This should let an equivariant model represent branched or incomplete symmetries that ordinary group-equivariant layers must discard.
Replace continuous stochastic-gradient updates by a flashing schedule with alternating ON phases, where gradients act normally, and OFF phases, where gradients are suppressed or weakened and controlled noise allows escape from local traps. Estimate directional asymmetry of the local loss basin from forward and backward probe distances, then set the flashing frequency using the ratchet resonance law so that noise-assisted transitions preferentially produce net progress toward lower loss.
Use the constraint matrix rank and nullity to set an explicit upper bound on the number of facial-reduction phases in an SDP layer representing structured binary decisions. Apply those phases before the main primal-dual solve, stopping after the rank–nullity budget and using the reduced face for all subsequent forward and backward computations.
Add coefficient-spreading and moment-calibration mechanisms to binary or sign-noised linear layers. For each output neuron, normalize its real-valued weights and penalize large normalized coordinates, so its signed preactivation obeys the paper's quadratic Gaussian approximation rather than the weaker linear bound. This should make activation scales more predictable and reduce training instability caused by highly concentrated binary projections.
Represent each neural prediction as a finite probability distribution and project it, under an optimal-transport cost, onto the set of distributions dominated by a teacher or target distribution in convex order. This enforces a global spread and risk relationship across all convex observables rather than adding separate variance, tail, and calibration penalties. Use a periodically refreshed projection during training and test whether it improves uncertainty calibration and robustness at equal…
For a coordinate-based neural network u_theta(x) solving a fully nonlinear second-order PDE, replace the raw quadratic-Hessian residual with the concave, homogeneous operator G(D_x^2 u_theta)=sqrt(sigma_2(D_x^2 u_theta)). Add differentiable barriers that keep the predicted Hessian inside the positive branch Gamma_2, preventing optimization from entering regions where the PDE operator is non-elliptic.
Treat the number K of minibatches between expensive control updates as a review period: the controlled neural dynamics use parameters or decisions computed at time nK and hold them fixed until (n+1)K. Scan K, estimate first and second finite differences of validation loss or episodic return, and use the resulting nonmonotone-to-convex or concave phase diagram to select an update frequency rather than assuming that more frequent updates are always better.
Add a task-aware error-protection code to a binary or low-cardinality latent representation. The encoder remains systematic, preserving the original latent coordinates, but appends repeated or parity coordinates computed from a linear task map so that latent states with different task values are separated by at least a chosen Hamming distance. Redundancy is allocated according to the rank of the task map rather than the full latent dimension.