A Homogeneous Tensor Framework for High-Order Trust-Region and Spherical Polynomial Optimization
arXiv:2607.24046
2026
Optimization
2 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper gives a constructive method for converting cubic Taylor models into homogeneous tensor contractions and solving the resulting spherical optimization problem with proximal alternating minimization. The transferable asset is the combination of unit-sphere constraints, closed-form block updates, monotone proximal decrease, and tensor contractions that avoid dense third-order tensors. The strongest neural-network applications are a cubic local optimizer for selected parameter blocks and a low-rank trilinear attention module whose latent factors are refined with stable spherical updates.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Replace the inner step of a neural optimizer with a safeguarded cubic local-model solve. Represent the cubic Taylor model as a homogeneous tensor in an augmented coordinate, solve proximal unit-sphere subproblems by alternating tensor contractions, decode a candidate step, and accept it only when the actual neural loss confirms the predicted decrease.
Useful7/10
Difficulty7/10
Novelty7/10
Unverified
2026
Build a low-rank trilinear attention module in which query, key, and value factors are constrained to the unit sphere and refined through a few proximal alternating sweeps. The proximal terms suppress factor oscillation and make each sweep improve a well-defined tensor interaction objective, offering a stable alternative to unconstrained tensor-power iterations.
Useful6/10
Difficulty5/10
Novelty7/10