Unverified 2026

Minkowski-Symmetry Activation Ellipsoid

Usefulness5/10

Difficulty6/10

Novelty4/10

Geometry Stability Accuracy Memory Mlp Transformer Quantization Initialization Convex analysis Geometry Optimization Linear algebra

Source paper: Symmetry-dependence in Rounding of a Convex Body arXiv:2608.30876 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace ordinary per-coordinate activation normalization or covariance whitening with a learned or frozen affine map based on a minimum-volume ellipsoid calibrated to the activation convex hull. Estimate the hull's Minkowski symmetry and use the bound $\rho=\sqrt{d/\alpha}$ to set a principled clipping radius, where $d$ is feature dimension and $\alpha$ is the measured symmetry. This should reduce worst-case clipping of asymmetric activation distributions without requiring a much larger quantization range.

Formulas

$$\operatorname{sym}(S):=\max\{\alpha\geq 0:\exists c\in S\text{ such that }-\alpha(S-c)\subseteq S-c\}.$$

$$E\subseteq S-c\subseteq\sqrt{\frac{d}{\operatorname{sym}(S)}}\,E.$$

$$E=\{Au:\|u\|_2\leq1\}=\{z:z^TQ^{-1}z\leq1\},\qquad Q=AA^T\succ0.$$

$$\alpha^*=\max_{\alpha,c,\lambda}\ \alpha\quad\text{s.t.}\quad -\alpha(x_j-c)=\sum_{k=1}^{m}\lambda_{jk}(x_k-c),\ \lambda_{jk}\geq0,\ \sum_{k=1}^{m}\lambda_{jk}=1\quad(j=1,\ldots,m).$$

Mathematical statement

For a full-dimensional convex body $S\subset\mathbb{R}^{d}$, the paper defines the Minkowski symmetry $\alpha=\operatorname{sym}(S):=\max\{a\geq0:\exists c\in S\text{ such that }-a(S-c)\subseteq S-c\}$; any maximizing $c$ is a Minkowski center. Its main theorem guarantees an origin-centered ellipsoid $E$ and center $c$ satisfying $E\subseteq S-c\subseteq\rho E$, with $\rho=\sqrt{d/\alpha}$. Represent $E$ as $E=\{Au:\|u\|_2\leq1\}$, equivalently $E=\{z:z^TQ^{-1}z\leq1\}$ for $Q=AA^T\succ0$. For a sampled activation set $X=\{x_j\}_{j=1}^m$, use its convex hull $S=\operatorname{conv}(X)$ as a finite approximation. Estimate $\alpha$ and $c$ by maximizing $a$ subject to $-a(x_j-c)\in\operatorname{conv}(X)$ for every sample $j$; each membership condition can be represented with simplex variables $\lambda_{jk}\geq0$, $\sum_k\lambda_{jk}=1$, and $-a(x_j-c)=\sum_k\lambda_{jk}(x_k-c)$. Then solve an ellipsoid program for $Q$, using the paper's containment relation as the target: the practical outer constraints are $(x_j-c)^TQ^{-1}(x_j-c)\leq\rho^2$ for all samples, while inner containment can be enforced approximately by support-point or halfspace constraints. The affine normalization is $y=A^{-1}(x-c)$, for which the calibrated region is contained in the Euclidean ball of radius $\rho$.

Implementation notes

Integrate this at the output of a chosen MLP or attention block, before quantization, residual addition, or a sensitive nonlinear activation. Collect an activation calibration matrix X of shape m by d from 1,000-10,000 representative training examples; use a random subsample or k-means representatives when m is large. First estimate the Minkowski center and symmetry. For each candidate center c and scalar alpha, solve the finite convex-hull membership constraints in the displayed formula using simplex variables lambda[j,k]; in an MVP, replace the full pairwise formulation with a projected-gradient search over c and alpha, checking membership with a linear program. Set rho=sqrt(d/max(alpha,epsilon)). Next solve a minimum-volume or regularized ellipsoid fit for Q: minimize -logdet(Q) plus a small ridge penalty, subject to (x[j]-c)^T Q^{-1}(x[j]-c)<=rho^2. Parameterize Q=LL^T and optimize the equivalent Cholesky variables if a generic SDP solver is unavailable. Store c and L^{-1}; at inference compute y=L^{-1}(x-c), clip y to the ball of radius rho (or coordinate-clip after rotation), then invert the transform if the following layer expects the original scale. The paper supplies the geometric guarantee; the engineer estimates alpha, c, Q, and the empirical coverage fraction from finite activations. Start with a 2-layer MLP on CIFAR-10 or a small Transformer on WikiText-2, comparing LayerNorm, diagonal min-max clipping, and ordinary covariance whitening. Measure validation accuracy or perplexity, fraction of clipped activations, INT8 calibration error, and added calibration/runtime cost. Success is lower clipping or quantization error at the same dynamic range, or equal error with a smaller range and at least 10% lower activation memory.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.