# Function-Separating Latent Code

- ID: 2827
- Canonical URL: https://synthcore.org/idea/2827/function-separating-latent-code
- API JSON: https://synthcore.org/api/idea/2827.json
- API Markdown: https://synthcore.org/api/idea/2827.md
- Verification status: unverified
- Source: [arXiv:2608.29389](https://arxiv.org/abs/2608.29389)
- Category: architecture
- Solves: stability, accuracy, memory
- ML areas: vae, quantization, loss, regularization
- Math tags: linear-algebra, combinatorics, coding-theory
- Ratings: usefulness 5/10; difficulty 5/10; novelty 6/10

## Idea description

Add a task-aware error-protection code to a binary or low-cardinality latent representation. The encoder remains systematic, preserving the original latent coordinates, but appends repeated or parity coordinates computed from a linear task map so that latent states with different task values are separated by at least a chosen Hamming distance. Redundancy is allocated according to the rank of the task map rather than the full latent dimension.

## Mathematical statement

The paper defines the function-separation distance as $\delta_{\operatorname{Enc}}(f)=\min\{d_H(\operatorname{Enc}(u),\operatorname{Enc}(v)):f(u)\neq f(v)\}$, with the requirement $\delta_{\operatorname{Enc}}(f)\ge d$. Here $u,v\in V=\mathbb F_q^k$ are messages, $f:V\to W$ is the task map, $\operatorname{Enc}:V\to\mathbb F_q^n$ is the code, and $d_H(a,b)=\operatorname{wt}(a-b)$ is Hamming distance. For a linear encoder, differences reduce the objective to $\delta_{\operatorname{Enc}}(f)=\min_{x\notin\ker f}\operatorname{wt}(\operatorname{Enc}(x))$: only directions outside the task kernel must have large weight. The paper identifies this quantity with the first relative generalized Hamming weight of the code relative to the encoded kernel and states that optimal free redundancy depends only on $\operatorname{rank}(f)=\dim\operatorname{Im}(f)=k-\dim\ker f$. A simple systematic neural construction is $\operatorname{Enc}_r(x)=(x,f(x),\ldots,f(x))$ with $r=d-1$ copies of $f(x)$; if $f(x)\neq0$, then $\operatorname{wt}(\operatorname{Enc}_r(x))\ge 1+r=d$, so every pair with different task values has separation at least $d$.

## Key formulas

- $$\delta_{\operatorname{Enc}}(f)=\min\{d_H(\operatorname{Enc}(u),\operatorname{Enc}(v)):f(u)\neq f(v)\},\qquad \delta_{\operatorname{Enc}}(f)\ge d.$$
- $$\delta_{\operatorname{Enc}}(f)=\min_{x\in V\setminus\ker f}\operatorname{wt}(\operatorname{Enc}(x))\quad\text{for linear }f\text{ and }\operatorname{Enc}.$$
- $$\operatorname{rank}(f)=\dim_{\mathbb F_q}\operatorname{Im}(f)=k-\dim\ker f.$$
- $$\operatorname{Enc}_r(x)=\bigl(x,f(x),\ldots,f(x)\bigr),\qquad r=d-1,\qquad \operatorname{wt}(\operatorname{Enc}_r(x))\ge 1+r=d\ \text{for }x\notin\ker f.$$

## Implementation notes

Use a binary bottleneck in a small VAE, classifier, or autoencoder. Let the encoder produce real logits a in R^k, form binary latent bits x=1[a>0] with a straight-through estimator, and learn a binary linear task map f(x)=A x mod 2 with A in F_2^{ell x k}; initialize A from a linear probe and jointly optimize it. At the code interface, construct z=(x,f(x),...,f(x)) with r=d-1 repetitions. During training, independently flip each bit of z with probability p, decode the corrupted code into the downstream prediction, and add BCE or cross-entropy on the task output plus a small code-consistency term that asks the decoder to reconstruct the uncorrupted task value. Pseudocode: compute a; x=STE-sign(a); y=f2(x,A); z=concat(x,[y]*r); z_noisy=BernoulliFlip(z,p); prediction=Decoder(z_noisy); loss=task_loss(prediction,target)+lambda*BCE(soft_decode(f2(x,A)),target_bits)+beta*rate_penalty. The paper's exact quantity is the minimum weight over x not in ker(f); estimate it empirically by enumerating all x for k<=16 or by sampling and targeted search for larger k. Rank(f) is computed by Gaussian elimination over F_2. Compare against an uncoded bottleneck with the same systematic bits and against ordinary repetition of all k bits at equal added coordinates. First test on MNIST or CIFAR-10 with k=16 binary latent bits, d=3, and bit-flip probabilities p in {0, .02, .05, .1}. Success means higher task accuracy under corruption at equal code length, or the same accuracy with fewer redundant coordinates; also measure whether protecting only rank(f) task directions beats uniform latent repetition.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.