# Boundary-Depth Sparsity

- ID: 1232
- Canonical URL: https://synthcore.org/idea/1232/boundary-depth-sparsity
- API JSON: https://synthcore.org/api/idea/1232.json
- API Markdown: https://synthcore.org/api/idea/1232.md
- Verification status: unverified
- Source: [arXiv:2607.21355](https://arxiv.org/abs/2607.21355)
- Category: regularization
- Solves: memory, speedup, generalization
- ML areas: cnn, regularization, pruning, inference-speedup
- Math tags: geometry, optimization, measure-theory, calculus-of-variations
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Add a nonlocal regularizer to nonnegative CNN feature maps that rewards activation-zero pixels lying deep inside a contiguous zero region. Unlike L1 sparsity, this penalizes isolated holes and favors block-like inactive areas that are more amenable to spatial skipping, channel gating, or structured pruning.

## Mathematical statement

The paper minimizes the functional J(u)=∫_Ω(1/2|∇u|²−fu)dx−∫_Z F(dist(x,∂Z))dx, where u:Ω→R_+ is the nonnegative field, Z={x∈Ω:u(x)=0} is its zero phase, ∂Z is the topological boundary of that phase, dist(x,A)=inf_{a∈A}|x−a| is Euclidean distance to a set, F:R→R_+ is nondecreasing, and f is a forcing term. The transferable mathematical asset is the depth-weighted zero-phase measure ∫_Z F(dist(x,∂Z))dx: a zero pixel receives little reward near the phase boundary and a larger reward when it lies deep inside a connected zero component. For a neural activation map a_c(i)≥0 on grid sites i∈Ω, use a soft zero indicator q_c(i)≈1[a_c(i)=0], a differentiable distance proxy d_c(i), and add R_depth=−λ∑_{c,i}q_c(i)F(d_c(i)) to the task loss. The adaptation is a smooth surrogate of the paper's topological zero set and distance functional, not a claim that the free-boundary existence theory directly applies to stochastic network training.

## Key formulas

- $$J(u)=\int_{\Omega}\left(\frac{1}{2}|\nabla u|^{2}-fu\right)\,dx-\int_{\{u=0\}}F\big(\mathrm{dist}(x,\partial\{u=0\})\big)\,dx.$$
- $$\mathrm{dist}(x,A):=\inf_{a\in A}|x-a|.$$
- $$\mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{task}}-\lambda\sum_{c}\sum_{i\in\Omega}q_c(i)F\big(d_c(i)\big),\qquad q_c(i)=\sigma\u200b\left(\frac{\tau-a_c(i)}{\varepsilon}\right).$$
- $$b_c(j)=q_c(j)(1-q_c(j))+\delta,\qquad d_c(i)=-\rho\log\left(\frac{\sum_{j\in\Omega}b_c(j)\exp(-\|i-j\|_2/\rho)}{\sum_{j\in\Omega}b_c(j)+\delta}\right).$$

## Implementation notes

Integrate the regularizer after a ReLU, Softplus, or absolute-value activation in a CNN, before the next convolution. Let a[b,c,h,w]≥0 be the feature tensor and let Ω be the H×W grid for one channel. First compute the soft zero membership q=sigmoid((τ−a)/ε), where τ is a small activation threshold, ε controls softness, and δ=10^-6 prevents division by zero. Approximate the zero-phase boundary by b=q(1−q)+δ; this is large for activations near the threshold and small in confidently active or confidently zero pixels. Compute the soft distance d at every pixel with the log-sum-exp soft minimum shown above, using a separable or truncated Gaussian neighborhood for efficiency. Choose F(d)=d^p with p=1 or 2, or F(d)=log(1+d), and add the negative depth reward to the task loss with coefficient λ. Pseudocode is: a=relu(conv(x)); q=sigmoid((τ−a)/ε); b=q*(1−q)+δ; d=soft_distance_to_boundary(b,rho); L=L_task−λ*mean(q*F(d)); backpropagate L and update weights. Anneal ε from 0.2 to 0.05 and λ from zero during the first few epochs to avoid immediate all-zero collapse. The paper contributes the depth-weighted zero-phase objective; the soft mask, boundary proxy, and distance transform are engineering approximations. Estimate sparsity, connected-component size, mean zero-pixel depth, validation accuracy, and hardware runtime. The first experiment should train ResNet-18 on CIFAR-10 with this term versus identical training with L1 activation regularization, group lasso, and no regularizer. At equal accuracy, success means fewer fragmented nonzeros, larger zero blocks, and measurable speedup after compiling block-sparse or spatial-skipping convolution; a useful target is 20–30% fewer active spatial blocks with less than 0.5 percentage-point accuracy loss.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.