Landscape analysis for shallow neural networks: Complete classification of critical points for cubic activation and affine target functions
arXiv:2607.15173
2026
Architecture
2 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper gives a constructive algebraic view of shallow networks with monomial activations: sums of shifted powers form a complete basis for univariate polynomials when enough distinct hidden pivots are available. The transferable asset is not merely the expressivity result, but the explicit Vandermonde solve, which can initialize polynomial subnetworks exactly instead of asking gradient descent to discover cancellations among large coefficients. The landscape classification also identifies a concrete failure mode: insufficient width or collapsed pivots causes unattained optima and diverging minimizing sequences, suggesting width-aware initialization and pivot-separation regularization for polynomial neural networks.
Ideas from this paper
Unverified
2026
Add a width- and degree-aware regularizer that prevents hidden polynomial neurons from collapsing to the same pivot. The paper's critical-point analysis says that non-global local minima and nontrivial saddles for cubic activation occur only when all pivots coincide, while global representations require at least d distinct active and visible pivots; the barrier directly targets this degeneracy.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Initialize a univariate polynomial-activation hidden layer to realize a prescribed polynomial exactly, rather than relying on gradient descent to learn the required cancellation between shifted monomials. This provides an analytically controlled starting point for polynomial MLPs, polynomial feature extractors, and teacher-to-student initialization when the desired local map is known or fitted from data.
Useful6/10
Difficulty4/10
Novelty7/10