Unverified 2026

All-Direction Frostman Representation

Usefulness6/10

Difficulty4/10

Novelty7/10

Geometry Generalization Stability Embedding Regularization Mlp Geometry Measure theory Harmonic analysis Dynamical systems

Source paper: Ergodic $\times p$-invariant measures on $\mathbb{T}^2$ with no dimension dropping projections arXiv:2608.29569 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Regularize a neural representation so that no one-dimensional projection places too much probability mass inside a narrow interval. This transfers the paper's uniform tube estimate into an anti-collapse constraint, making representations robust to adversarial directions and preventing hidden features from becoming effectively low-dimensional.

Formulas

$$\dim(\mu):=\inf\{\dim_H A:\mu(A)>0\},\qquad \dim(\pi_u\mu)=\dim(\mu)\ \text{for every }u\in S^1.$$

$$\mu\big(\{z\in\mathbb{R}^2:\operatorname{dist}(z,L)\leq w\}\big)\leq Cw^s\quad\text{for every line }L,\ 0<w\leq 1.$$

$$(\pi_u\mu)([a-w,a+w])\leq C'(2w)^s,\qquad \pi_u(z)=u^\top z.$$

$$\mathcal{L}_{\mathrm{tube}}=\frac{1}{|\mathcal{U}|}\sum_{u\in\mathcal{U}}\sum_{w\in\mathcal{W}}\left[\log q_{u,w}^{\max}-\log C-s\log(2w)\right]_+^2,\quad q_{u,w}^{\max}=\max_a\frac{1}{B}\sum_{i=1}^B\sigma\!\left(\frac{w-|u^\top z_i-a|}{\tau}\right).$$

Mathematical statement

Let $\mu$ be the distribution of a two-dimensional hidden representation $z=f_\theta(x)$, let $u\in S^1$ be a unit projection direction, and let $\pi_u(z)=u^\top z$. The paper's mechanism is the uniform tube/Frostman estimate $\mu(\{z:\operatorname{dist}(z,L)\leq w\})\leq Cw^s$ for every line $L$, width $0<w\leq 1$, and exponent $0<s<1$. Equivalently, every projected measure obeys $(\pi_u\mu)([a-w,a+w])\leq C'(2w)^s$ for every $u$, center $a$, and $w$. For a minibatch $z_i$, estimate the left side with a differentiable kernel count and penalize violations of $\log q(u,a,w)\leq \log C+s\log(2w)$, where $q$ is the empirical projected small-ball mass. The property doing the work is uniformity over $u$ and $a$, not merely good behavior for randomly sampled projections.

Implementation notes

Integrate this after the encoder and before the task head. Use a two-dimensional projection head $z_i=g(f_\theta(x_i))$; retain the task loss, and optimize $\mathcal{L}=\mathcal{L}_{task}+\lambda\mathcal{L}_{tube}$. At each step, sample 32--128 unit directions $u$ uniformly on the circle, use logarithmically spaced widths $\mathcal{W}=\{2^{-2},2^{-3},\ldots,2^{-7}\}$ after batchwise normalization of z, and estimate the worst interval center by evaluating centers at the projected batch values plus a fixed grid. Pseudocode: compute z; for each u compute r=u^T z; for each w and center a compute the soft count q using the sigmoid in the displayed formula; take the maximum over a; accumulate the hinge-square penalty; backpropagate task loss plus penalty. The paper supplies the target power law and its all-directions interpretation; the batch counts, constant C, smoothing temperature $\tau$, and exponent s are empirical design choices. Start with s=0.7, $\lambda\in\{10^{-3},10^{-2},10^{-1}\}$, and $\tau=0.05w$. The first cheap test is a two-layer MLP on MNIST or CIFAR-10 with cross-entropy, comparing the regularizer against no regularizer and standard variance normalization. Probe 512 directions, including adversarially optimized directions, and report the maximum projected interval mass versus w. The prediction is a log-log slope at least s-0.1 over the unsaturated scales for the regularized model, while the baseline has at least one direction with a substantially smaller slope; increasing $\lambda$ should reduce the worst-direction mass monotonically until task accuracy begins to degrade. Also test robustness after adding a small perturbation to inputs: the regularized representation should show a smaller increase in worst-direction concentration.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.