# Pisot-Orbit Deterministic JL Layer

- ID: 2221
- Canonical URL: https://synthcore.org/idea/2221/pisot-orbit-deterministic-jl-layer
- API JSON: https://synthcore.org/api/idea/2221.json
- API Markdown: https://synthcore.org/api/idea/2221.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.13078](https://arxiv.org/abs/2608.13078)
- Category: architecture
- Solves: speedup, memory, scalability
- ML areas: embedding, retrieval, inference-speedup, moe-routing
- Math tags: probability, dynamical-systems, spectral-theory, number-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a dense random projection used before retrieval, classification, or expert routing with a publicly reproducible matrix generated by a Pisot beta-transformation orbit. Search over a small public seed and sampling gap to select one matrix that preserves the calibration set's pairwise distances, then freeze it for training and inference. The projection removes random-matrix storage and makes the same embedding transform exactly reproducible across servers or proof systems.

## Mathematical statement

Let beta>1 be a Pisot number, meaning an algebraic integer whose non-principal Galois conjugates have modulus strictly less than one. Define the beta-transformation on the unit interval by T_beta(u)=beta u-floor(beta u), and let u_t=T_beta^t(u_0) be its orbit from seed u_0. Convert consecutive orbit values into centered, unit-variance coefficients z_t=(u_t-mu)/sigma, where mu and sigma are the empirical mean and standard deviation of a long calibration orbit. Form a projection P in R^{m x d} by P_{r,i}=z_{s+(r-1)g+i}/sqrt(m), with seed offset s and sampling gap g; rescale P once so that the squared-norm estimator targets the original squared norm. The spectral-gap argument gives geometrically decaying correlations, so the estimator has bias bounded by |E[||Px||_2^2]-||x||_2^2| <= C rho^g ||x||_2^2 for 0<rho<1, and variance bounded by Var(||Px||_2^2) <= (V_0/m)||x||_2^4. Here C depends on the observable and the beta-map spectral gap, rho is its correlation-decay factor, V_0 is dimension-independent under the paper's concentration hypothesis, m is the projected dimension, and x is the input vector. For a finite set of N calibration vectors, search over seed s and gap g and minimize maximum pairwise distortion. In a finite-field implementation, replace real orbit arithmetic with the exact recurrence induced by beta's minimal polynomial and perform arithmetic modulo the circuit field.

## Key formulas

- $$T_{\beta}(u)=\beta u-\lfloor \beta u\rfloor,\qquad u_t=T_{\beta}^{t}(u_0),\qquad z_t=\frac{u_t-\mu}{\sigma}.$$
- $$P_{r,i}=\frac{z_{s+(r-1)g+i}}{\sqrt{m}},\qquad y=Px,\qquad x\in\mathbb{R}^{d},\ y\in\mathbb{R}^{m}.$$
- $$\left|\mathbb{E}\,\|Px\|_2^2-\|x\|_2^2\right|\le C\rho^{g}\|x\|_2^2,\qquad \operatorname{Var}\!\left(\|Px\|_2^2\right)\le \frac{V_0}{m}\|x\|_2^4,\quad 0<\rho<1.$$
- $$(s^*,g^*)=\arg\min_{s,g}\ \max_{1\le a<b\le N}\left|\frac{\|P_{s,g}(c_a-c_b)\|_2^2}{\|c_a-c_b\|_2^2}-1\right|,$$ where $c_a,c_b$ are calibration centroids.

## Implementation notes

(1) Integration point: insert the fixed projection immediately after a high-dimensional embedding or pooled transformer representation and before cosine retrieval, nearest-centroid classification, or an MoE router. For a 768-dimensional embedding and m=128, use a 128x768 deterministic matrix. During training, freeze this projection initially; optionally add a learned diagonal scale after projection so the representation can compensate for mild anisotropy without learning a full dense matrix. (2) Pseudocode: choose beta, seed u0, offset s, and gap g; iterate u=T_beta(u) and collect at least m+d orbit values; compute mu and sigma on a long burn-in orbit; set P[r,i]=z[s+(r-1)*g+i]/sqrt(m); compute a scalar alpha from calibration data using alpha^2=(sum_a ||c_a||_2^2)/(sum_a ||P*c_a||_2^2); output y=alpha*P*x. Enumerate several hundred seeds and gaps, evaluate maximum relative pairwise centroid distortion, and freeze the best matrix. (3) Compute from the mathematics: the beta orbit, centered coefficients, projection entries, and the proposed geometric dependence on g. Estimate empirically: mu, sigma, alpha, the best seed, pairwise distortion, and V0 from held-out vectors. Measure correlation decay by estimating |corr(z_t,z_{t+k})| and fitting an exponential envelope A*rho^k. For a proof-oriented prototype, implement the orbit with fixed-point or finite-field arithmetic and compare projection hashes across machines. (4) First cheap experiment: use CIFAR-10 class embeddings or a small language-model retrieval benchmark; compare against Gaussian JL, sparse JL, and a learned linear projection at equal m. Measure retrieval recall, centroid accuracy, maximum pairwise distortion, projection-generation time, stored bytes, and downstream training loss. Success means matching Gaussian-JL accuracy within measurement noise while being exactly reproducible, reducing generation or commitment cost, observing approximately geometric bias reduction with larger g, and observing variance proportional to 1/m.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a deterministic Pisot-orbit JL layer using the golden-ratio beta transformation, empirical centering/scaling, scalar norm calibration, seed/gap search, reproducibility hashing, and Gaussian-JL comparison. Correlations decayed strongly after a short transient and calibration search improved worst pairwise distortion from 1.891 to 0.522, but the global correlation fit was non-decaying (rho≈1.0001) and norm-estimator variance scaled with slope -0.72 rather than the predicted -1.0. Under the required mechanism criterion, this is not a validated win.

### Mechanism check

- Verdict: Built a deterministic Pisot-orbit JL layer using the golden-ratio beta transformation, empirical centering/scaling, scalar norm calibration, seed/gap search, reproducibility hashing, and Gaussian-JL comparison. Correlations decayed strongly after a short transient and calibration search improved worst pairwise distortion from 1.891 to 0.522, but the global correlation fit was non-decaying (rho≈1.0001) and norm-estimator variance scaled with slope -0.72 rather than the predicted -1.0. Under the required mechanism criterion, this is not a validated win.
- Confidence: 9/10
- Limitations: Only a small synthetic Gaussian-vector experiment was tested; no CIFAR, retrieval, classifier, router, learned post-scale, sparse-JL comparison, GPU benchmark, exact finite-field implementation, or proof of the stated spectral-gap bounds was performed. The calibration search uses the same small calibration set for selection and evaluation, so its distortion improvement may overfit.

## Artifacts

- [compare.py](https://synthcore.org/code/882/compare.py)
- [experiment.py](https://synthcore.org/code/882/experiment.py)
- [report.md](https://synthcore.org/code/882/report.md)
- [results.json](https://synthcore.org/code/882/results.json)
- [Download all files as ZIP](https://synthcore.org/download/882)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.