Simplex-Complexity Depth Diagnostic
Implementation & benchmark of arXiv:2607.03815 — A simplex-based measure of symmetry
Source paper: A simplex-based measure of symmetry arXiv:2607.03815 ⓘ · analyzed Aug 29, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Estimate the simplex-based ratio of a target or learned convex piecewise-linear polytope and use the theorem \(\rho_\Delta(P)\le 2^d-1\) to choose a minimum useful ReLU depth. During training, monitor whether the learned polytope is approaching a high-\(\rho\) target; if it is, widen the model without increasing depth only when the diagnostic indicates that depth is the bottleneck.
Formulas
Mathematical statement
For a fixed reference simplex \(\Delta\subset\mathbb{R}^n\), define \(\lambda_K(L)\) as the smallest \(\lambda\ge 0\) for which there exists a translation vector \(t\in\mathbb{R}^n\) satisfying \(L\subseteq t+\lambda K\), where \(K\) is either \(\Delta\) or \(-\Delta\). The paper defines \(\rho_\Delta(L)=\lambda_{-\Delta}(L)/\lambda_\Delta(L)\), which is invariant under scaling and obeys \(1/n\le \rho_\Delta(L)\le n\) for every non-point compact convex set \(L\subset\mathbb{R}^n\). For polytopes generated by depth-\(d\) compositions of Minkowski sum and convex hull of a union, the paper proves the sharp obstruction \(\rho_\Delta(P)\le 2^d-1\). Here \(P\) is the convex hull of sampled subgradient, gradient, or latent feature vectors of a convex piecewise-linear network. Therefore a target polytope with measured ratio \(r\) requires the diagnostic depth lower bound \(d\ge\lceil\log_2(r+1)\rceil\) under the paper's compositional model. The ratio is estimated by two small linear programs: minimize \(\lambda\) over \(t,\lambda\) subject to every vertex \(p_i\) of \(P\) lying in \(t+\lambda\Delta\), and repeat with \(-\Delta\).
Implementation notes
Integrate this as an architecture-selection and monitoring tool for convex ReLU MLPs, not as a per-example loss in the first MVP. Use a small convex piecewise-linear network with nonnegative output weights, or use the convex hull of feature-gradient vectors from an unconstrained ReLU MLP as an empirical proxy. (1) Choose dimension n equal to the feature or output dimension being analyzed, and fix a numerically stable simplex, for example \(\Delta=\operatorname{conv}(0,e_1,\ldots,e_n)\), where \(e_i\) are standard basis vectors. (2) Collect a batch of vectors \(p_i\): for a convex scalar network, these can be input gradients \(\nabla_x f(x_i)\); alternatively use penultimate-layer features after centering and projecting to n dimensions. Remove duplicates and compute their convex hull, retaining vertices \(p_i\). (3) Solve the LP for \(\lambda_\Delta(P)\): minimize \(\lambda\) over translation \(t\in\mathbb{R}^n\) and \(\lambda\ge0\), subject to \(p_i\in t+\lambda\Delta\) for all vertices. Encode membership using the simplex barycentric inequalities; solve a second LP with the vertices of \(-\Delta\) to obtain \(\lambda_{-\Delta}(P)\). Form \(\widehat\rho=\lambda_{-\Delta}/\lambda_\Delta\), adding a small denominator epsilon. (4) Set the initial depth to \(\lceil\log_2(\widehat\rho+1)\rceil\), compare against shallower and deeper baselines, and log \(\widehat\rho\) throughout training. The theorem-derived quantity is only the depth obstruction; the sampled hull and LP values are empirical estimates and are not guaranteed to equal the network's exact global polytope. First experiment: fit randomly generated convex piecewise-linear functions in dimension n=4 using width-matched ReLU MLPs of depths 1--5. Compare parameter count, validation error, and optimization time against the diagnostic depth. Success means high-\(\rho\) targets show a sharp error/fit improvement once the predicted depth is reached, while low-\(\rho\) targets do not benefit comparably from extra depth.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.