Topology-Guided Capacity Allocation

Integrate this as a periodic architecture controller around an ordinary MLP, CNN, or transformer encoder, operating on the hidden-state tensor at each block rather than inside the forward pass. First choose a fixed validation subset of 512 examples and cache activations every 250 to 1000 optimizer steps. For each layer, flatten the batch and token or spatial dimensions into a point cloud, optionally subsample 512 points, normalize each feature dimension, and build a k-nearest-neighbor graph with k=10 or 15. For the MVP, compute \(\beta_0^{(\ell)}\) as the number of connected components across a distance-threshold sweep; use a persistent-homology library to obtain \(\beta_1^{(\ell)}\) if affordable. Set \(w_0=1,w_1=0.25\), compute \(B(\ell)\), the running minimum \(\tilde B(\ell)\), all \(\Delta(\ell)\), and COM exactly as in the formulas. At a scheduled intervention point, identify \(\ell^*=\arg\max\Delta(\ell)\). If COM is below target \(\tau=0.6(L+1)\), widen that layer by \(\delta h=25\%\) using neuron replication plus small Gaussian perturbations, or insert one residual block immediately after it. Reinitialize only the new parameters and continue training; impose a parameter budget so capacity is moved rather than unlimitedly added. If COM is above target and validation accuracy is unchanged, prune 10% of parameters in layers after the dominant drop. The mathematical quantities are computed directly from persistent homology; kNN size, subsampling, threshold discretization, target COM, and intervention frequency are empirical hyperparameters. Start with a 6-layer width-128 MLP on MNIST, comparing uniform widening, random-layer widening, and this controller at equal final parameter count. Measure test accuracy, COM trajectory, parameter count, and wall-clock overhead. Success means later COM and lower reconstruction error or higher accuracy at the same parameter budget, with topology computation accounting for less than 10% of training time.