# All-Direction Frostman Representation

- ID: 2837
- Canonical URL: https://synthcore.org/idea/2837/all-direction-frostman-representation
- API JSON: https://synthcore.org/api/idea/2837.json
- API Markdown: https://synthcore.org/api/idea/2837.md
- Verification status: unverified
- Source: [arXiv:2608.29569](https://arxiv.org/abs/2608.29569)
- Category: geometry
- Solves: generalization, stability
- ML areas: embedding, regularization, mlp
- Math tags: geometry, measure-theory, harmonic-analysis, dynamical-systems
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Regularize a neural representation so that no one-dimensional projection places too much probability mass inside a narrow interval. This transfers the paper's uniform tube estimate into an anti-collapse constraint, making representations robust to adversarial directions and preventing hidden features from becoming effectively low-dimensional.

## Mathematical statement

Let \(\mu\) be the distribution of a two-dimensional hidden representation \(z=f_\theta(x)\), let \(u\in S^1\) be a unit projection direction, and let \(\pi_u(z)=u^\top z\). The paper's mechanism is the uniform tube/Frostman estimate \(\mu(\{z:\operatorname{dist}(z,L)\leq w\})\leq Cw^s\) for every line \(L\), width \(0<w\leq 1\), and exponent \(0<s<1\). Equivalently, every projected measure obeys \((\pi_u\mu)([a-w,a+w])\leq C'(2w)^s\) for every \(u\), center \(a\), and \(w\). For a minibatch \(z_i\), estimate the left side with a differentiable kernel count and penalize violations of \(\log q(u,a,w)\leq \log C+s\log(2w)\), where \(q\) is the empirical projected small-ball mass. The property doing the work is uniformity over \(u\) and \(a\), not merely good behavior for randomly sampled projections.

## Key formulas

- $$\dim(\mu):=\inf\{\dim_H A:\mu(A)>0\},\qquad \dim(\pi_u\mu)=\dim(\mu)\ \text{for every }u\in S^1.$$
- $$\mu\big(\{z\in\mathbb{R}^2:\operatorname{dist}(z,L)\leq w\}\big)\leq Cw^s\quad\text{for every line }L,\ 0<w\leq 1.$$
- $$(\pi_u\mu)([a-w,a+w])\leq C'(2w)^s,\qquad \pi_u(z)=u^\top z.$$
- $$\mathcal{L}_{\mathrm{tube}}=\frac{1}{|\mathcal{U}|}\sum_{u\in\mathcal{U}}\sum_{w\in\mathcal{W}}\left[\log q_{u,w}^{\max}-\log C-s\log(2w)\right]_+^2,\quad q_{u,w}^{\max}=\max_a\frac{1}{B}\sum_{i=1}^B\sigma\!\left(\frac{w-|u^\top z_i-a|}{\tau}\right).$$

## Implementation notes

Integrate this after the encoder and before the task head. Use a two-dimensional projection head \(z_i=g(f_\theta(x_i))\); retain the task loss, and optimize \(\mathcal{L}=\mathcal{L}_{task}+\lambda\mathcal{L}_{tube}\). At each step, sample 32--128 unit directions \(u\) uniformly on the circle, use logarithmically spaced widths \(\mathcal{W}=\{2^{-2},2^{-3},\ldots,2^{-7}\}\) after batchwise normalization of z, and estimate the worst interval center by evaluating centers at the projected batch values plus a fixed grid. Pseudocode: compute z; for each u compute r=u^T z; for each w and center a compute the soft count q using the sigmoid in the displayed formula; take the maximum over a; accumulate the hinge-square penalty; backpropagate task loss plus penalty. The paper supplies the target power law and its all-directions interpretation; the batch counts, constant C, smoothing temperature \(\tau\), and exponent s are empirical design choices. Start with s=0.7, \(\lambda\in\{10^{-3},10^{-2},10^{-1}\}\), and \(\tau=0.05w\). The first cheap test is a two-layer MLP on MNIST or CIFAR-10 with cross-entropy, comparing the regularizer against no regularizer and standard variance normalization. Probe 512 directions, including adversarially optimized directions, and report the maximum projected interval mass versus w. The prediction is a log-log slope at least s-0.1 over the unsaturated scales for the regularized model, while the baseline has at least one direction with a substantially smaller slope; increasing \(\lambda\) should reduce the worst-direction mass monotonically until task accuracy begins to degrade. Also test robustness after adding a small perturbation to inputs: the regularized representation should show a smaller increase in worst-direction concentration.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.