{
 "artifacts": null,
 "category": "memory",
 "description": "Use the measured power-exponential covering dimension of an activation manifold to choose the growth rate of latent codebooks, prototypes, or quantization resolution. The goal is to avoid spending parameters on a representation whose attainable resolution exceeds the intrinsic covering complexity.",
 "formulas_latex": [
  "$$\\underline{\\operatorname{dim}}_{\\mathrm{M},\\exp}(\\mathscr C)=\\liminf_{\\varepsilon\\downarrow0}\\frac{\\log_2\\log_2\\mathcal N(\\mathscr C,\\varepsilon)}{\\log_2(1/\\varepsilon)}=s_*.$$",
  "$$\\mathcal N(\\mathscr C,\\varepsilon)\\approx\\exp\\!\\left(C\\varepsilon^{-s_*}\\right),\\qquad \\log_2\\mathcal N(\\mathscr C,\\varepsilon)\\asymp\\varepsilon^{-s_*}.$$",
  "$$\\varepsilon(K)=\\left(\\frac{C}{\\log K}\\right)^{1/s}\\quad\\text{or equivalently}\\quad K(\\varepsilon)=\\exp\\!\\left(C\\varepsilon^{-s}\\right).$$"
 ],
 "id": 2895,
 "implementation": "Apply this to a VQ-VAE, residual vector quantizer, product quantizer, or token bottleneck. Collect activations \\(z=f_\\theta(x)\\) from a calibration set and estimate \\(\\widehat N(r)\\) with greedy farthest-point covering: start from one activation, repeatedly add the point farthest from the current centers, and count centers needed for radius \\(r\\). Fit \\(\\widehat s\\) and \\(\\widehat C\\) by robust regression of \\(\\log\\log(\\widehat N(r)+e)\\) against \\(\\log(1/r)\\), using only radii where the fit is stable. Given a fixed code budget \\(K\\), set the target quantization error to \\(\\varepsilon_K=(\\widehat C/\\log(K+1))^{1/\\widehat s}\\). During training, increase codebook size only when the measured reconstruction error is below \\(0.5\\varepsilon_K\\) for several evaluations; if it remains above \\(2\\varepsilon_K\\), allocate more codes or increase residual stages. The mathematical scaling is taken from the covering-number phase transition; \\(\\widehat C\\), finite-sample radius range, and the engineering thresholds are empirical. Begin with CIFAR-10 and ImageNet-100 using a VQ-VAE with 256, 512, and 1024 total codes, comparing a fixed-code baseline and a standard perplexity-triggered growth rule at equal decoder capacity. Measure reconstruction error, downstream linear-probe accuracy, active-code perplexity, total codebook memory, and training steps to reach a target distortion. Success is equal distortion with fewer codes or faster convergence, especially when the activation geometry changes during training.",
 "math_summary": "The relevant object is the covering number \\(\\mathcal N(\\mathscr C,\\varepsilon)\\), the smallest number of radius-\\(\\varepsilon\\) metric balls covering the activation set \\(\\mathscr C\\). Its lower power-exponential dimension is \\(s_*\\), defined by \\(s_* = \\liminf_{\\varepsilon\\downarrow0}\\log_2\\log_2\\mathcal N(\\mathscr C,\\varepsilon)/\\log_2(1/\\varepsilon)\\). When the liminf is an actual limit and \\(\\mathcal N(\\mathscr C,\\varepsilon)\\approx\\exp(C\\varepsilon^{-s_*})\\), the paper's phase transition says that resolution \\(\\varepsilon\\) requires approximately \\(\\log_2\\mathcal N(\\mathscr C,\\varepsilon)\\asymp\\varepsilon^{-s_*}\\) bits, while rates beyond this scaling cannot represent more than a null subset under the critical measure. For a vector-quantized bottleneck with \\(K\\) codes, invert this relation as \\(\\varepsilon(K)\\approx(C^{-1}\\log K)^{-1/s}\\), where \\(s\\) is an empirical estimate of \\(s_*\\).",
 "math_tags": [
  "metric-geometry",
  "approximation-theory",
  "information-theory"
 ],
 "ml_areas": [
  "vae",
  "quantization",
  "memory",
  "embedding"
 ],
 "paper": {
  "arxiv_id": "2608.30549",
  "arxiv_url": "https://arxiv.org/abs/2608.30549",
  "summary_what_math_gives_to_ml": "The paper provides a general way to turn covering-number growth into a probability measure whose mass is exponentially small in every sufficiently small metric ball. This suggests a representation-learning regularizer: instead of merely maximizing pairwise distances, enforce that the empirical distribution of embeddings has a controlled small-ball profile, with exponent calibrated from the intrinsic power-exponential dimension of the data or feature set. A second transfer is to use the same dimension estimate to allocate codebook capacity and quantization resolution in bottleneck models. Both ideas are empirical adaptations of the paper's covering-number and critical-measure results, not direct consequences for finite minibatches.",
  "title": "Maximally Spread Out Measures and Implications for Phase Transitions in Approximation Theory",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 5
 },
 "solves": [
  "sample-efficiency",
  "scalability",
  "memory"
 ],
 "title": "Dimension-Calibrated Bottleneck Schedule",
 "url": "https://synthcore.org/idea/2895/dimension-calibrated-bottleneck-schedule",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}