Discrete eigenvalue optimization from entropic smoothing and first-order methods
arXiv:2608.27024
2026
Architecture
1 ideas extracted · analyzed Aug 29, 2026
What the math gives to ML
The paper formulates combinatorial selection through the smallest eigenvalue of an affine positive-semidefinite matrix, with M(x)=C+sum_i x_i m_i m_i^T and binary decisions x_i. This objective rewards selections whose vectors span all directions well, rather than merely selecting high-norm or individually strong candidates. A direct neural-network transfer is spectral-diversity pruning or expert/channel selection: construct candidate activation signatures m_i, relax binary selection to a capped simplex, optimize a differentiable entropic soft minimum eigenvalue, then round to the desired budget. The resulting module should retain complementary channels or experts and can improve accuracy at a fixed parameter, FLOP, or routing budget.
Ideas from this paper
✗ Mechanism failed
2026
Replace magnitude-based channel or expert pruning with a subset-selection objective that maximizes the weakest direction in the candidates' activation span. Relax the binary mask to continuous gates, optimize an entropic soft minimum eigenvalue, and round the gates to retain a fixed number of channels or experts. This should preserve diverse representations and reduce redundant feature directions.
Useful7/10
Difficulty5/10
Novelty7/10