Unverified 2026

Minimum-entropy symmetry noise

Implementation & benchmark of arXiv:2607.29020 — Entropic Symmetrization Resistance

Usefulness6/10
Difficulty6/10
Novelty7/10

Source paper: Entropic Symmetrization Resistance arXiv:2607.29020 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Insert an additive noise layer on a discrete latent space G, choosing the noise distribution g so that the convolved latent distribution f*g is symmetric under inversion while keeping H(g) small. For binary or nearly binary categorical latents, use the paper's explicit sparse cyclic-group construction instead of uniform augmentation, preserving symmetry with substantially less randomization.

Formulas

$$H(g)=-\sum_{x\in\operatorname{supp}(g)}g(x)\log g(x)$$
$$(f*g)(x)=\sum_{y\in G}f(y)g(x-y),\qquad (f*g)(x)=(f*g)(-x)\ \text{for every }x\in G$$
$$f_k(-1)=1-p,\quad f_k(1)=p,\quad k=2l+1,\quad \delta=\frac{2p-1}{kp-l}=\frac{2p-1}{2lp+p-l},\quad g_k(0)=1-l\delta$$
$$\min_{g\in\Delta_k}H(g)\quad\text{subject to}\quad \sum_{y\in\mathbb Z_k}f(y)g(x-y)=\sum_{y\in\mathbb Z_k}f(y)g(-x-y)\quad\text{for every }x\in\mathbb Z_k$$

Mathematical statement

For a countable additive group G, f and g are probability mass functions and convolution is defined by (f*g)(x) = sum_y f(y)g(x-y). A distribution g is a symmetrizer of f when (f*g)(x) = (f*g)(-x) for every x in G. The injected randomness is Shannon entropy H(g) = -sum_x g(x) log g(x). For G = Z_k with odd k = 2l+1, the paper considers f_k(-1) = 1-p and f_k(1) = p with p greater than 1/2. It defines delta = (2p-1)/(kp-l) and g_k(0) = 1-l delta, while assigning probability delta to exactly l nonzero residues and zero to the other l residues using the stated residue pattern. This sparse construction makes f_k*g_k inversion-symmetric and has H(g_k) tending to zero as p decreases to 1/2. The ML adaptation minimizes H(g) subject to the linear symmetry constraints, whose feasible solutions often lie on the boundary of the probability simplex and therefore have sparse support.

Implementation notes

Integrate this as a stochastic augmentation layer on a categorical latent, class index, token cluster, or graph-node label. Represent each latent state by an element of Z_k. For the exact two-point path, identify the two dominant states, map them to -1 and +1, estimate p from the minibatch histogram, choose an odd k, compute l = (k-1)/2 and delta = (2p-1)/(kp-l), then construct g with g(0) = 1-l delta and the sparse residue pattern from the paper. Sample y from g and replace latent state x by (x+y) modulo k before the downstream network. For a general categorical distribution f, form a linear constraint matrix A whose row for x contains the coefficients of sum_y f(y)g(x-y) minus sum_y f(y)g(-x-y). Optimize g on the simplex using projected gradient or Frank-Wolfe, minimizing entropy subject to A g = 0; use a small positive floor during optimization, then threshold tiny entries and renormalize. Compute f from an exponential moving average of minibatch histograms, while the symmetry constraints and entropy are exact. Start with MNIST or CIFAR-10, a k = 7 categorical bottleneck, and compare no augmentation, uniform Z_7 noise, and minimum-entropy noise at equal FLOPs. Record validation accuracy, calibration, latent reconstruction error, entropy H(g), and residual max_x absolute value of (f*g)(x) minus (f*g)(-x). Success means matching or exceeding uniform-noise accuracy with lower entropy and fewer distinct transformations per example.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.