Unverified 2026

Multiscale Cross-Branch Co-Dimension Regularizer

Implementation & benchmark of arXiv:2606.30998 — Distributional results for the shortest distance between trajectories of different dynamics

Usefulness5/10

Difficulty5/10

Novelty6/10

Geometry Accuracy Generalization Stability Regularization Embedding World model Metric geometry Measure theory Probability

Source paper: Distributional results for the shortest distance between trajectories of different dynamics arXiv:2606.30998 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Measure the local geometric compatibility of q latent distributions produced by different views, augmentations, environments, or trajectory models using the paper's co-dimension. Penalize excessive cross-branch co-dimension over a range of radii while preserving per-branch variance and covariance rank to prevent representation collapse.

Formulas

$$C_q(\mu_1,\ldots,\mu_q):=\lim_{r\to 0}\frac{\log\int_{\mathcal M}\prod_{i=2}^{q}\mu_i(B_r(x))\,d\mu_1(x)}{(q-1)\log r},\qquad B_r(x)=\{y\in\mathcal M:d(x,y)\le r\}.$$

$$I(r):=\int_{\mathcal M}\prod_{i=2}^{q}\mu_i(B_r(x))\,d\mu_1(x),\qquad \widehat C_q=\frac{\sum_{k=1}^{K}(\log r_k-\overline{\log r})(\log \widehat I(r_k)-\overline{\log \widehat I})}{(q-1)\sum_{k=1}^{K}(\log r_k-\overline{\log r})^2}.$$

$$\widehat I_\tau(r)=\frac{1}{n_1}\sum_{a=1}^{n_1}\prod_{i=2}^{q}\left[\frac{1}{n_i}\sum_{b=1}^{n_i}\exp\left(-\frac{\|z^{(1)}_a-z^{(i)}_b\|_2^2}{2\tau r^2}\right)\right],\qquad \mathcal L_{\mathrm{overlap}}=\sum_{k=1}^{K}w_k\,\widehat C_q(r_k)^2.$$

Mathematical statement

Definition 2.1 defines the co-dimension of probability measures $\mu_1,\ldots,\mu_q$ on a common metric space $\mathcal M$ by $C_q(\mu_1,\ldots,\mu_q)=\lim_{r\to0}\frac{\log\int_{\mathcal M}\prod_{i=2}^{q}\mu_i(B_r(x))\,d\mu_1(x)}{(q-1)\log r}$. Here $B_r(x)=\{y\in\mathcal M:d(x,y)\le r\}$ is the radius-$r$ metric ball, $\mu_i(B_r(x))$ is the local mass of distribution $i$ around anchor $x$, and $q$ is the number of distributions. The numerator is the logarithm of the expected product of local masses from the other $q-1$ distributions around an anchor sampled from $\mu_1$; the denominator extracts its small-radius power-law exponent. The paper's Remark 2.2 states that this quantity is symmetric in all measures despite the asymmetric-looking integral. For neural representations, replace $\mu_i$ by the empirical latent distribution of branch or environment $i$, estimate local masses with soft balls, and fit the slope of $\log I(r)$ versus $\log r$ across radii. A lower slope indicates stronger cross-distribution local overlap, but an anti-collapse term is required because a point mass would trivially produce slope zero.

Implementation notes

1. Integration point: use a shared encoder $f_\theta$ and create $q$ views of each sample, or feed the same initial state through $q$ environment or trajectory models. Obtain normalized latent batches $z^{(i)}_b=f_\theta(x^{(i)}_b)\in\mathbb R^d$. Apply the regularizer to embeddings before the prediction head, not to logits. 2. Pseudocode: choose logarithmically spaced radii $r_1<\cdots<r_K$ from batch pairwise-distance quantiles. For every radius, compute $s_{a,i}(r)=n_i^{-1}\sum_b\exp(-\|z^{(1)}_a-z^{(i)}_b\|^2/(2\tau r^2))$, then $\widehat I(r)=n_1^{-1}\sum_a\prod_{i=2}^q s_{a,i}(r)$. Set $y_k=\log(\widehat I(r_k)+\epsilon)$, fit the least-squares slope $\widehat C_q$ of $y_k$ against $(q-1)\log r_k$, and add $\lambda(\widehat C_q-C_{\mathrm{target}})^2$ to the task loss. 3. The paper supplies the local-mass integral and power-law slope interpretation. The implementation estimates $I(r)$ with Gaussian soft balls, estimates the slope across radii, and estimates radii from current batch distance quantiles. Add a variance term $\sum_j\max(0,\gamma-\operatorname{std}(z_{:,j}))^2$ and optionally covariance decorrelation, preventing the overlap term from collapsing all embeddings. 4. First experiment: train a small SimCLR or BYOL encoder on CIFAR-10 with two or four augmentations, comparing the baseline against the same model plus this regularizer. Use batches of 256, $K=5$ radii between the 10th and 50th pairwise-distance percentiles, and sweep $\lambda\in\{10^{-3},10^{-2},10^{-1}\}$. Measure linear-probe accuracy, augmentation consistency, per-coordinate standard deviation, covariance rank, and estimated co-dimension. Success means improved linear-probe accuracy or robustness at equal compute without reduced latent variance; a secondary signal is faster alignment between views while maintaining nontrivial representation rank.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.