ML: Sparsity

Machine-learning ideas tagged Sparsity in the ML taxonomy of the Math2NN corpus.

2 ideas found

Status: Unverified ML: Sparsity Clear all

Unverified 2026

Shifted-Mask Defect Regularization

Represent a learned sparse attention or routing pattern as a graph and penalize its second-moment defect, which measures distance from a shifted family and therefore from nested, threshold-like neighborhoods. At inference, optionally replace the learned mask by a nearby shifted mask to obtain more structured sparse indexing and predictable routing patterns.

Useful5/10

Difficulty6/10

Novelty9/10

Regularization Memory Scalability Stability

Paper: Stability of Shifted Complexes via the Second-Moment Defect of the Up-Laplacian arXiv:2608.15358

Unverified 2026

Ensemble-Averaged Sparse Graph Search

Replace a binary neural connectivity mask by independent Bernoulli edge probabilities and optimize a deterministic expected message-passing objective before discretization. The resulting module can search sparse GNN edges or expert-to-token routes without repeatedly sampling many discrete architectures during training.

Useful5/10

Difficulty5/10

Novelty5/10

Architecture Scalability Accuracy Memory

Paper: Learning LDPC codes with quantized density evolution over relaxed protographs arXiv:2607.08484