Pisot-Separated Multiscale Codes

1. Integration point: use this as a replacement for ordinary learned positional embeddings, product-quantization codebooks, or the hidden-state memory in a small recurrent model. Choose a finite digit set \(D\), for example learned vectors constrained to unit norm, and represent a token or memory state by the sum of scale-specific digits \(z=\sum_{k=0}^{K-1}\theta^k d_k\). Parameterize \(\theta\) as a fixed Pisot number such as the golden ratio, or learn the coefficients of a monic polynomial while penalizing roots outside the unit disk. 2. Pseudocode: initialize digit vectors \(D\); for each input, predict digits \(d_0,\ldots,d_{K-1}\); compute z = sum(theta**k * d[k] for k in range(K)); feed z to the downstream network; every validation interval enumerate or randomly sample code pairs and compute delta_K; add lambda_sep * relu(margin-delta_K)^2 to the loss. 3. Computed from the paper: the Pisot criterion and the contracting-conjugate bound. Estimated empirically: minimum separation, collision rate, task loss, and gradient norm. 4. First cheap experiment: train a two-layer MLP on CIFAR-10 or a synthetic hierarchical classification task with ordinary learned embeddings versus golden-ratio, integer-scale, and non-Pisot \(\theta=\sqrt{2}\) codes, using equal parameter counts and \(K=8,16,32\). Measure pairwise code separation and nearest-neighbor collision rate as \(K\) increases. The quantitative prediction is that the Pisot model has a substantially slower decline of \(\delta_K\) and collision rate than the non-Pisot model; a falsification criterion is failure to obtain at least a 2x lower collision rate at \(K=32\), or no separation advantage after controlling for code norm.