# Anchored-Box Coverage Regularizer

- ID: 1322
- Canonical URL: https://synthcore.org/idea/1322/anchored-box-coverage-regularizer
- API JSON: https://synthcore.org/api/idea/1322.json
- API Markdown: https://synthcore.org/api/idea/1322.md
- Verification status: unverified
- Source: [arXiv:2607.23571](https://arxiv.org/abs/2607.23571)
- Category: regularization
- Solves: generalization, sample-efficiency, accuracy
- ML areas: regularization, embedding, vae
- Math tags: discrepancy, probability, combinatorics
- Ratings: usefulness 5/10; difficulty 4/10; novelty 6/10

## Idea description

Add a minibatch regularizer that measures how uniformly latent representations cover the unit cube by comparing empirical mass in lower-orthant boxes with a target distribution. Rather than estimating the full star discrepancy, sample boxes and coordinate subsets, and use soft indicators so the term is differentiable. This should discourage representation collapse and improve coverage of rare regions without requiring pairwise repulsion between all examples.

## Mathematical statement

The paper defines the star discrepancy of an N-point multiset P={x_1,...,x_N} in [0,1)^d as D_N^*(P)=sup_{y in [0,1]^d}|N^{-1} sum_n 1_{[0,y)}(x_n)-vol([0,y))}, where [0,y)=product_j[0,y_j). Its Anchored-box transfer lemma states that if f=c_0+sum_{ell=1}^L c_ell 1_{A_ell}, with each A_ell an anchored box [0,y^(ell)), then |N^{-1}sum_n f(x_n)-integral f| <= D_N^*(P) sum_ell |c_ell|. The paper also uses h(t)=(3 1_{[0,1/3)}(t)-1)/sqrt(2), for which integral h=0, integral h^2=1, and h(t)^2=(3/2)1_{[0,1/3)}(t)+1/2. For a coordinate subset S of size k, phi_S(x)=product_{j in S}h(x_j) expands into anchored-box indicators with total coefficient mass 2^{3k/2}, so its empirical mean is bounded by 2^{3k/2}D_N^*(P). We use sampled anchored-box errors as a differentiable proxy for discrepancy and low-order phi_S correlations as diagnostics.

## Key formulas

- $$D_N^{\ast}(P):=\sup_{y\in[0,1]^d}\left|\frac{1}{N}\sum_{n=1}^{N}\mathbf{1}_{[0,y)}(x_n)-\operatorname{vol}([0,y))\right|.$$
- $$\left|\frac{1}{N}\sum_{n=1}^{N}f(x_n)-\int_{[0,1)^d}f(x)\,dx\right|\le D_N^{\ast}(P)\sum_{\ell=1}^{L}|c_\ell|,\qquad f=c_0+\sum_{\ell=1}^{L}c_\ell\mathbf{1}_{A_\ell},\ A_\ell=[0,y^{(\ell)}).$$
- $$h(t)=\frac{3\mathbf{1}_{[0,1/3)}(t)-1}{\sqrt{2}},\qquad \int_0^1h(t)\,dt=0,\quad \int_0^1h(t)^2\,dt=1,\quad h(t)^2=\frac{3}{2}\mathbf{1}_{[0,1/3)}(t)+\frac{1}{2}.$$
- $$\phi_S(x)=\prod_{j\in S}h(x_j)=2^{-|S|/2}\sum_{T\subseteq S}3^{|T|}(-1)^{|S|-|T|}\mathbf{1}_{A_T}(x),\qquad \left|\frac1N\sum_n\phi_S(x_n)\right|\le 2^{3|S|/2}D_N^*(P).$$

## Implementation notes

1. Integration point: attach the regularizer to an encoder output, embedding layer, or generator. Map each representation z in R^m to u=sigmoid((z-mu)/tau) in (0,1)^d, using d=8--32 selected coordinates or a learned random projection. Compute it on each minibatch B={u_i}_{i=1}^N. 2. Pseudocode: sample R random box corners y_r in (0,1)^d and coordinate masks M_r; compute q_r=(1/N) sum_i product_{j in M_r} sigmoid((y_{rj}-u_{ij})/tau), target p_r=product_{j in M_r} y_{rj}, and L_box=(1/R)sum_r w_r(q_r-p_r)^2. Optimize L_total=L_task+lambda L_box. The product over M_r is a soft version of the anchored-box indicator; coordinates outside M_r are unrestricted. Anneal tau from 0.2 to 0.03 and clip gradients. 3. The paper supplies the discrepancy definition, the anchored-box transfer inequality, and the h-product expansion. The implementation estimates the supremum empirically by random corners; report max_r |q_r-p_r| and low-order diagnostics |N^{-1}sum_i phi_S(u_i)| for |S|=1,2,3. Maintain an exponential-memory pool of corners with the largest violations and resample half the corners from that pool. 4. First experiment: train a small VAE or 2-layer MLP classifier on MNIST or CIFAR-10 with a 16-dimensional sigmoid bottleneck. Compare no regularizer, pairwise repulsion, covariance whitening, and L_box at equal FLOPs. Measure nearest-neighbor collapse, occupancy of a random-projection grid, held-out likelihood or classification accuracy, and sampled discrepancy. Success means lower maximum box error and fewer empty latent regions without harming task loss; target at least a 10% reduction in collapse metrics at equal accuracy.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.