# Pisot-Separated Multiscale Codes

- ID: 2154
- Canonical URL: https://synthcore.org/idea/2154/pisot-separated-multiscale-codes
- API JSON: https://synthcore.org/api/idea/2154.json
- API Markdown: https://synthcore.org/api/idea/2154.md
- Verification status: unverified
- Source: [arXiv:2608.11867](https://arxiv.org/abs/2608.11867)
- Category: architecture
- Solves: stability, memory, accuracy
- ML areas: embedding, quantization, memory, transformer
- Math tags: dynamical-systems, geometry, linear-algebra, spectral-theory
- Ratings: usefulness 5/10; difficulty 6/10; novelty 9/10

## Idea description

Replace an unconstrained geometric multiscale codebook by features generated from a finite digit set and a Pisot scale factor. The contracting algebraic-conjugate directions should suppress near-collisions between representations at different scales, producing a discretely separated hierarchy that can be used for embeddings, recurrent memory, or quantized transformer states.

## Mathematical statement

A real number \(\theta>1\) is Pisot if it is an algebraic integer and every algebraic conjugate \(\theta_j\neq\theta\) satisfies \(|\theta_j|<1\). The paper's mechanism states that, under the relevant self-similar Delone hypotheses, uniform discreteness of a point pattern \(\Lambda\) is equivalent to \(\theta\) being Pisot. A neural analogue uses a finite digit set \(D\subset\mathbb{R}^d\) and hierarchical codes \(z=\sum_{k=0}^{K-1}\theta^k d_k\), with \(d_k\in D\). The conjugate-coordinate map \(\Phi(z)=(\sum_k\theta_1^k d_k,\ldots,\sum_k\theta_m^k d_k)\) remains bounded in nonprincipal coordinates because \(|\theta_j|<1\). The engineering certificate is a positive minimum separation \(\delta_K=\min_{z\neq z'}\|z-z'\|\) over valid codes; the hypothesis is that Pisot scales maintain a nonvanishing or slowly decaying \(\delta_K\), whereas non-Pisot scales exhibit increasingly frequent near-collisions as depth \(K\) grows.

## Key formulas

- $$\theta>1\text{ is Pisot}\quad\Longleftrightarrow\quad \theta\text{ is an algebraic integer and }|\theta_j|<1\ \text{for every conjugate }\theta_j\neq\theta.$$
- $$z(d_0,\ldots,d_{K-1})=\sum_{k=0}^{K-1}\theta^k d_k,\qquad d_k\in D,$$
- $$\Phi(z)=\left(\sum_{k=0}^{K-1}\theta_1^k d_k,\ldots,\sum_{k=0}^{K-1}\theta_m^k d_k\right),\qquad \left\|\sum_{k=0}^{K-1}\theta_j^k d_k\right\|\leq \frac{\max_{d\in D}\|d\|}{1-|\theta_j|}.$$
- $$\delta_K(\theta)=\min_{d\neq d'}\left\|\sum_{k=0}^{K-1}\theta^k(d_k-d'_k)\right\|;\qquad \text{test whether }\inf_{K\leq K_{\max}}\delta_K(\theta)>0\text{ for Pisot }\theta.$$

## Implementation notes

1. Integration point: use this as a replacement for ordinary learned positional embeddings, product-quantization codebooks, or the hidden-state memory in a small recurrent model. Choose a finite digit set \(D\), for example learned vectors constrained to unit norm, and represent a token or memory state by the sum of scale-specific digits \(z=\sum_{k=0}^{K-1}\theta^k d_k\). Parameterize \(\theta\) as a fixed Pisot number such as the golden ratio, or learn the coefficients of a monic polynomial while penalizing roots outside the unit disk. 2. Pseudocode: initialize digit vectors \(D\); for each input, predict digits \(d_0,\ldots,d_{K-1}\); compute `z = sum(theta**k * d[k] for k in range(K))`; feed `z` to the downstream network; every validation interval enumerate or randomly sample code pairs and compute `delta_K`; add `lambda_sep * relu(margin-delta_K)^2` to the loss. 3. Computed from the paper: the Pisot criterion and the contracting-conjugate bound. Estimated empirically: minimum separation, collision rate, task loss, and gradient norm. 4. First cheap experiment: train a two-layer MLP on CIFAR-10 or a synthetic hierarchical classification task with ordinary learned embeddings versus golden-ratio, integer-scale, and non-Pisot \(\theta=\sqrt{2}\) codes, using equal parameter counts and \(K=8,16,32\). Measure pairwise code separation and nearest-neighbor collision rate as \(K\) increases. The quantitative prediction is that the Pisot model has a substantially slower decline of \(\delta_K\) and collision rate than the non-Pisot model; a falsification criterion is failure to obtain at least a 2x lower collision rate at \(K=32\), or no separation advantage after controlling for code norm.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.