# RG-monotone representation compression

- ID: 2612
- Canonical URL: https://synthcore.org/idea/2612/rg-monotone-representation-compression
- API JSON: https://synthcore.org/api/idea/2612.json
- API Markdown: https://synthcore.org/api/idea/2612.md
- Verification status: unverified
- Source: [arXiv:2608.20268](https://arxiv.org/abs/2608.20268)
- Category: architecture
- Solves: memory, generalization, stability
- ML areas: mlp, cnn, transformer, distillation, pruning
- Math tags: renormalization, information-theory, linear-algebra, statistical-mechanics
- Ratings: usefulness 5/10; difficulty 6/10; novelty 6/10

## Idea description

Interpret successive neural representations as an RG flow and constrain coarse-graining layers to remove unstable or redundant information monotonically. The paper reports monotonic decrease of an effective central charge along measurement-induced RG flows; a neural analogue can use a measurable information-complexity proxy and reject compression steps that increase it while preserving task-relevant information.

## Mathematical statement

The paper invokes an effective-central-charge theorem stating that a suitable Casimir effective central charge decreases along the measurement-induced RG flow: \(c_{eff}^{IR}\le c_{eff}^{UV}\). The exact central charge is not directly available for generic neural representations, so define an operational proxy \(C(h)=B^{-1}\log\det(I+\alpha K_h)\), where \(h\in\mathbb{R}^{B\times d}\) is a batch of representations, \(K_h\) is its centered normalized Gram matrix, \(B\) is batch size, and \(\alpha>0\) controls sensitivity. A compression map \(h_{l+1}=g_l(h_l)\) is accepted only if \(C(h_{l+1})\le C(h_l)+\epsilon\), while a task-loss constraint prevents trivial constant representations. The mechanism being tested is monotone complexity flow, not the claim that this proxy equals the field-theoretic central charge.

## Key formulas

- $$c_{eff}^{IR}\le c_{eff}^{UV}$$
- $$h_{l+1}=g_l(h_l),\qquad C(h)=\frac{1}{B}\log\det\!\left(I+\alpha K_h\right)$$
- $$\mathcal{L}=\mathcal{L}_{task}+\lambda\sum_l\left[\max\left(0,C(h_{l+1})-C(h_l)+\epsilon\right)\right]^2$$

## Implementation notes

1. Integration point: insert learnable bottleneck or pooling layers between blocks of a CNN, MLP, or transformer. Compute a batch Gram matrix from the hidden states at each adjacent pair of layers and add the monotonic-flow penalty to the training loss. For transformers, use tokenwise hidden states after attention and after the feed-forward block, optionally averaging C over tokens. 2. Pseudocode: forward the minibatch and cache h[0],...,h[L]; for each layer l, center h[l] over the batch, normalize each feature, form K_l=h[l]h[l]^T/B, compute C_l=logdet(I+alpha*K_l+delta I)/B using a Cholesky factorization, and add lambda*relu(C_{l+1}-C_l+epsilon)^2 to cross-entropy. Backpropagate through the Cholesky estimate, or use a detached C_l for a cheap controller that increases bottleneck strength when C rises. 3. Computed from the paper: the direction of the RG inequality, namely nonincrease from UV to IR. Estimated empirically: C, task mutual-information proxies, rank, calibration, and accuracy after pruning or quantization. 4. First cheap experiment: train a 6-layer MLP and a small CNN on CIFAR-10 with and without a 128-to-32 bottleneck at each third layer. Sweep lambda and epsilon, recording C_l by epoch and test accuracy under input noise. The quantitative prediction is that accepted compression schedules produce C_{l+1}-C_l<=epsilon for at least 95 percent of layers and batches, while an unconstrained network will show upward jumps; models with monotone C should degrade more gradually under post-training rank truncation. Falsify the idea if the penalty cannot enforce monotonicity without sacrificing more than 2 accuracy points, or if C has no predictive relationship with robustness or compression loss.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.