# Minkowski-Symmetry Activation Ellipsoid

- ID: 2908
- Canonical URL: https://synthcore.org/idea/2908/minkowski-symmetry-activation-ellipsoid
- API JSON: https://synthcore.org/api/idea/2908.json
- API Markdown: https://synthcore.org/api/idea/2908.md
- Verification status: unverified
- Source: [arXiv:2608.30876](https://arxiv.org/abs/2608.30876)
- Category: geometry
- Solves: stability, accuracy, memory
- ML areas: mlp, transformer, quantization, initialization
- Math tags: convex-analysis, geometry, optimization, linear-algebra
- Ratings: usefulness 5/10; difficulty 6/10; novelty 4/10

## Idea description

Replace ordinary per-coordinate activation normalization or covariance whitening with a learned or frozen affine map based on a minimum-volume ellipsoid calibrated to the activation convex hull. Estimate the hull's Minkowski symmetry and use the bound \(\rho=\sqrt{d/\alpha}\) to set a principled clipping radius, where \(d\) is feature dimension and \(\alpha\) is the measured symmetry. This should reduce worst-case clipping of asymmetric activation distributions without requiring a much larger quantization range.

## Mathematical statement

For a full-dimensional convex body \(S\subset\mathbb{R}^{d}\), the paper defines the Minkowski symmetry \(\alpha=\operatorname{sym}(S):=\max\{a\geq0:\exists c\in S\text{ such that }-a(S-c)\subseteq S-c\}\); any maximizing \(c\) is a Minkowski center. Its main theorem guarantees an origin-centered ellipsoid \(E\) and center \(c\) satisfying \(E\subseteq S-c\subseteq\rho E\), with \(\rho=\sqrt{d/\alpha}\). Represent \(E\) as \(E=\{Au:\|u\|_2\leq1\}\), equivalently \(E=\{z:z^TQ^{-1}z\leq1\}\) for \(Q=AA^T\succ0\). For a sampled activation set \(X=\{x_j\}_{j=1}^m\), use its convex hull \(S=\operatorname{conv}(X)\) as a finite approximation. Estimate \(\alpha\) and \(c\) by maximizing \(a\) subject to \(-a(x_j-c)\in\operatorname{conv}(X)\) for every sample \(j\); each membership condition can be represented with simplex variables \(\lambda_{jk}\geq0\), \(\sum_k\lambda_{jk}=1\), and \(-a(x_j-c)=\sum_k\lambda_{jk}(x_k-c)\). Then solve an ellipsoid program for \(Q\), using the paper's containment relation as the target: the practical outer constraints are \((x_j-c)^TQ^{-1}(x_j-c)\leq\rho^2\) for all samples, while inner containment can be enforced approximately by support-point or halfspace constraints. The affine normalization is \(y=A^{-1}(x-c)\), for which the calibrated region is contained in the Euclidean ball of radius \(\rho\).

## Key formulas

- $$\operatorname{sym}(S):=\max\{\alpha\geq 0:\exists c\in S\text{ such that }-\alpha(S-c)\subseteq S-c\}.$$
- $$E\subseteq S-c\subseteq\sqrt{\frac{d}{\operatorname{sym}(S)}}\,E.$$
- $$E=\{Au:\|u\|_2\leq1\}=\{z:z^TQ^{-1}z\leq1\},\qquad Q=AA^T\succ0.$$
- $$\alpha^*=\max_{\alpha,c,\lambda}\ \alpha\quad\text{s.t.}\quad -\alpha(x_j-c)=\sum_{k=1}^{m}\lambda_{jk}(x_k-c),\ \lambda_{jk}\geq0,\ \sum_{k=1}^{m}\lambda_{jk}=1\quad(j=1,\ldots,m).$$

## Implementation notes

Integrate this at the output of a chosen MLP or attention block, before quantization, residual addition, or a sensitive nonlinear activation. Collect an activation calibration matrix X of shape m by d from 1,000-10,000 representative training examples; use a random subsample or k-means representatives when m is large. First estimate the Minkowski center and symmetry. For each candidate center c and scalar alpha, solve the finite convex-hull membership constraints in the displayed formula using simplex variables lambda[j,k]; in an MVP, replace the full pairwise formulation with a projected-gradient search over c and alpha, checking membership with a linear program. Set rho=sqrt(d/max(alpha,epsilon)). Next solve a minimum-volume or regularized ellipsoid fit for Q: minimize -logdet(Q) plus a small ridge penalty, subject to (x[j]-c)^T Q^{-1}(x[j]-c)<=rho^2. Parameterize Q=LL^T and optimize the equivalent Cholesky variables if a generic SDP solver is unavailable. Store c and L^{-1}; at inference compute y=L^{-1}(x-c), clip y to the ball of radius rho (or coordinate-clip after rotation), then invert the transform if the following layer expects the original scale. The paper supplies the geometric guarantee; the engineer estimates alpha, c, Q, and the empirical coverage fraction from finite activations. Start with a 2-layer MLP on CIFAR-10 or a small Transformer on WikiText-2, comparing LayerNorm, diagonal min-max clipping, and ordinary covariance whitening. Measure validation accuracy or perplexity, fraction of clipped activations, INT8 calibration error, and added calibration/runtime cost. Success is lower clipping or quantization error at the same dynamic range, or equal error with a smaller range and at least 10% lower activation memory.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.