# Topology-Guided Capacity Allocation

- ID: 1728
- Canonical URL: https://synthcore.org/idea/1728/topology-guided-capacity-allocation
- API JSON: https://synthcore.org/api/idea/1728.json
- API Markdown: https://synthcore.org/api/idea/1728.md
- Verification status: unverified
- Source: [arXiv:2608.02816](https://arxiv.org/abs/2608.02816)
- Category: architecture
- Solves: accuracy, generalization, scalability
- ML areas: mlp, training-dynamics, initialization
- Math tags: topology, algebraic-topology, geometry
- Ratings: usefulness 5/10; difficulty 6/10; novelty 7/10

## Idea description

Use the layer at which persistent connected components and holes disappear to allocate capacity nonuniformly across a network. If representations simplify much earlier than desired, widen the responsible layers or insert an additional block; if simplification is excessively delayed, avoid spending parameters there. This turns persistent-homology COM into an actionable architecture-search signal rather than a post-hoc visualization.

## Mathematical statement

For layer \(\ell\), let \(\beta_k^{(\ell)}\) be the Betti number in homological degree \(k\), computed from the persistent homology of the layer's activation point cloud; \(\beta_0\) counts connected components and \(\beta_1\) counts one-dimensional holes. Choose nonnegative weights \(w_k\) and define the weighted topological complexity \(B(\ell)=\sum_{k\in\mathcal K}w_k\beta_k^{(\ell)}\), where \(\mathcal K\) is typically \(\{0,1\}\). Because raw Betti estimates can fluctuate, define the irreversible running minimum \(\tilde B(\ell)=\min_{j\leq\ell}B(j)\). The simplification at layer \(\ell\) is \(\Delta(\ell)=\tilde B(\ell-1)-\tilde B(\ell)\), and the normalized event distribution is \(p(\ell)=\Delta(\ell)/D\), with total drop \(D=\sum_\ell\Delta(\ell)\). The center of mass is \(\mathrm{COM}=\sum_{\ell=1}^{L+1}\ell p(\ell)=\frac{\sum_{\ell=1}^{L+1}\ell\Delta(\ell)}{\sum_{\ell=1}^{L+1}\Delta(\ell)}\). A low COM means topological simplification occurs early. In the adapted controller, compare COM to a target \(\tau\) and add capacity near the earliest large drop: \(\ell^*=\arg\max_\ell\Delta(\ell)\).

## Key formulas

- $$B(\ell)=\sum_{k\in\mathcal K}w_k\beta_k^{(\ell)},\qquad \tilde B(\ell)=\min_{j\leq \ell}B(j)$$
- $$\Delta(\ell)=\tilde B(\ell-1)-\tilde B(\ell),\qquad p(\ell)=\frac{\Delta(\ell)}{D},\qquad D=\sum_\ell\Delta(\ell)$$
- $$\mathrm{COM}=\sum_{\ell=1}^{L+1}\ell p(\ell)=\frac{\sum_{\ell=1}^{L+1}\ell\Delta(\ell)}{\sum_{\ell=1}^{L+1}\Delta(\ell)}$$
- $$\ell^*=\arg\max_\ell\Delta(\ell),\qquad \text{if }\mathrm{COM}<\tau:\; h_{\ell^*}\leftarrow h_{\ell^*}+\delta h\text{ or insert a block after }\ell^*$$

## Implementation notes

Integrate this as a periodic architecture controller around an ordinary MLP, CNN, or transformer encoder, operating on the hidden-state tensor at each block rather than inside the forward pass. First choose a fixed validation subset of 512 examples and cache activations every 250 to 1000 optimizer steps. For each layer, flatten the batch and token or spatial dimensions into a point cloud, optionally subsample 512 points, normalize each feature dimension, and build a k-nearest-neighbor graph with k=10 or 15. For the MVP, compute \(\beta_0^{(\ell)}\) as the number of connected components across a distance-threshold sweep; use a persistent-homology library to obtain \(\beta_1^{(\ell)}\) if affordable. Set \(w_0=1,w_1=0.25\), compute \(B(\ell)\), the running minimum \(\tilde B(\ell)\), all \(\Delta(\ell)\), and COM exactly as in the formulas. At a scheduled intervention point, identify \(\ell^*=\arg\max\Delta(\ell)\). If COM is below target \(\tau=0.6(L+1)\), widen that layer by \(\delta h=25\%\) using neuron replication plus small Gaussian perturbations, or insert one residual block immediately after it. Reinitialize only the new parameters and continue training; impose a parameter budget so capacity is moved rather than unlimitedly added. If COM is above target and validation accuracy is unchanged, prune 10% of parameters in layers after the dominant drop. The mathematical quantities are computed directly from persistent homology; kNN size, subsampling, threshold discretization, target COM, and intervention frequency are empirical hyperparameters. Start with a 6-layer width-128 MLP on MNIST, comparing uniform widening, random-layer widening, and this controller at equal final parameter count. Measure test accuracy, COM trajectory, parameter count, and wall-clock overhead. Success means later COM and lower reconstruction error or higher accuracy at the same parameter budget, with topology computation accounting for less than 10% of training time.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.