Machine-learning ideas tagged Graph nn in the ML taxonomy of the Math2NN corpus.
Regularize probability-valued network outputs in the square-root representation rather than directly penalizing density curvature. This suppresses sharp oscillations while avoiding the severe scaling of derivative penalties involving \(\nabla\rho/\rho\) near vacuum regions.
Replace an ordinary elementwise interaction between two feature matrices by a noncommutative functional-calculus layer \(\varphi(A,B)\), where \(A\) and \(B\) are Hermitian channel operators that need not commute. Add a soft penalty on \([A,B]=AB-BA\), and use a Besov-smooth parameterization of \(\varphi\) so that perturbations are controlled in Schatten \(p\)-norm for \(p\leq2\). This creates a principled matrix interaction module that can remain stable when feature operators or graph…
Replace independent top-k expert decisions by a global fractional routing problem that enforces token-side and expert-side capacities together with an additional diversity constraint represented by a partition or laminar matroid. Use the resulting Hall-type deficiency certificate to identify overloaded token subsets and penalize the actual structural cause of routing failure rather than relying only on an aggregate load-balancing loss.
Use the finite-order characterization to learn a nonlinear similarity function for token, patch, or graph-node Gram matrices while preserving PSD by construction or by a differentiable certificate loss. This creates a kernelized attention or graph-readout mechanism in which nonlinear affinity transformations cannot introduce indefinite similarity geometry.
Replace a costly global PSD constraint on a learned symmetric similarity or covariance matrix with the paper's 2-local PSD constraint. Every 2-by-2 principal submatrix is guaranteed valid, preventing excessively large pairwise correlations while avoiding eigendecomposition or Cholesky factorization of the full matrix.
Represent a learned sparse attention or routing pattern as a graph and penalize its second-moment defect, which measures distance from a shifted family and therefore from nested, threshold-like neighborhoods. At inference, optionally replace the learned mask by a nearby shifted mask to obtain more structured sparse indexing and predictable routing patterns.
Replace pointwise pair interactions between mesh cells by quadrature of the interaction kernel over the full Cartesian product of the two cells. Decompose each cell pair into convex-hull pieces and apply a Duffy-like radial transformation so the coincidence singularity is confined to one quadrature coordinate, allowing fixed Gauss-Jacobi or adaptive quadrature to produce smooth, low-variance interaction features.
Apply a trainable scalar gate entrywise to a Min/Max structured affinity or covariance matrix while enforcing that the gate is nonnegative, nondecreasing, and convex. This preserves Loewner ordering on the structured cone and avoids unconstrained elementwise nonlinearities that can destroy PSD or order relations.
Replace a Euclidean feature-space metric by a short-time heat-kernel/Wasserstein metric and use it to precondition updates or penalize distortions of local neighborhoods. The first-order correction is a Ricci-curvature term, while the second-order residual captures curvature variation and quadratic curvature effects that ordinary diffusion smoothing misses.
Insert a fixed or partially learnable equivariant change-of-basis module into a spherical or SO(3)-equivariant network. At each angular frequency \(\ell\), the module maps the line selected by the line-bundle quantization to the line selected by the Grauert-tube quantization, allowing the network to represent both holomorphic/base-local and geodesic-flow-adapted features without breaking rotation equivariance.
Represent a directed interaction graph by a Laurent-polynomial Euler-like matrix and use its evaluation as a signed message-passing or attention-mixing operator. During dynamic rewiring, require the new graph representation to preserve the associated bilinear form up to the congruence transformation induced by the change of basis, so equivalent routings produce equivalent hidden states.
Replace an unconstrained categorical or multilabel output head with a graph-supported distribution over feasible independent sets. Given neural logits, assign probability proportional to the exponential of the total logit of each selected vertex, so incompatible vertices can never be jointly active. Use exact junction-tree inference for decomposable graphs with small treewidth, and compare against post-hoc masking or penalty-based constraint enforcement.
Normalize every higher-order simplicial message-passing or diffusion block using the spectral radius of a lower-order up-Laplacian, rather than estimating a separate radius for each order. The paper's monotonicity theorem guarantees that this shared bound is conservative for all higher orders, enabling stable explicit updates with one spectral calibration.
Represent a feature field with positive channel amplitudes and penalize violations of the paper's system-wide relative-variation bound. Unlike per-channel total variation, the penalty constrains only aggregate channel mass, allowing channels to exchange mass through signed or non-cooperative mixing while keeping the overall representation stable. The method is most natural for intermediate CNN maps, positive SSM states, or sequence embeddings indexed by a coordinate with meaningful local…
Replace a sharp graph-Laplacian spectral filter with a Bochner–Riesz filter whose smoothness exponent increases when the graph contains regions with different effective dimensions. Estimate the largest local dimension and dimension gap from neighborhood growth, then choose the exponent above both the classical spectral threshold and the asymmetric obstruction threshold. This should suppress unstable high-frequency mixing in heterogeneous graphs while preserving more low-frequency signal than…
Regularize a learned GNN adjacency so that its random walk mixes rapidly, reducing graph bottlenecks and isolated regions that make information propagation inefficient. Use a thresholded penalty rather than minimizing Kemeny's constant to zero, because excessively fast mixing can produce oversmoothing.
Replace independent Bernoulli branch dropout in a tree-structured mixture or hierarchical MLP with connectivity gates sampled from a q<1 wired random-cluster model. The q<1 law provides conditional negative association across branches, so increasing statistics of disjoint branches have nonpositive covariance; this should reduce redundant expert activation while preserving structured stochastic exploration.
Precompute a two-valued edge labeling of every input tree so that adjacent vertices have different weighted incident-edge sums. Feed the edge labels and resulting vertex signatures into message passing as deterministic symmetry breakers. This can distinguish branches that otherwise produce identical initial representations without adding trainable parameters or random node identifiers.
Replace dense token-to-token attention with attention over connected token groups, called polymers, while forbidding nearby polymers from being simultaneously selected. Each candidate group receives an exponentially decaying size and boundary penalty, and the layer sums or samples only compatible collections of groups. The construction should create structured sparsity and prevent redundant overlapping attention regions.
Parameterize a learned 3-state transition operator as a product of at most seven elementary row-stochastic matrices rather than learning its nine entries independently. Each factor performs one convex pull-in of row i toward row j, so every intermediate and final matrix remains row-stochastic and the layer has a sparse, bounded-depth interpretation.
Represent the hidden state of a recurrent or implicit neural block by a convex reachable set and encode its recursive constraints as containment inequalities rather than unrolling a fixed number of steps. Eliminate the set variables to obtain the smallest representable invariant set, which can be used as a tighter robustness certificate, a training regularizer, or a principled initialization for equilibrium solvers.
Construct a symmetric feature-interaction or Jacobian matrix A_theta whose desired rank is t, then regularize its t-th compound matrix toward rank one. This transfers the paper's identity that a rank-t matrix has a rank-one t-th compound, while the rank-one factor encodes Plucker coordinates of the kernel subspace.
Assign each of K entity or token types an integer code from a B_{2,\Delta}-set A, so every unordered pair {i,j} produces a unique and margin-separated scalar code a_i+a_j. Use this code as a compact symmetric pair feature for graph edges, attention biases, or pairwise relation MLPs, avoiding collisions that occur when ordinary low-dimensional additive encodings are quantized or hashed.
Add a nonlocal logarithmic-Laplacian penalty to intermediate spatial feature maps or ordered token embeddings. Unlike a standard graph or image Laplacian, the kernel uses scale-free weights proportional to |z|^{-n} and includes a local compensation term, allowing multiscale feature smoothing without simply forcing nearby features to become identical.