# Exact Involutive Sprugnoli Mixer

- ID: 1672
- Canonical URL: https://synthcore.org/idea/1672/exact-involutive-sprugnoli-mixer
- API JSON: https://synthcore.org/api/idea/1672.json
- API Markdown: https://synthcore.org/api/idea/1672.md
- Verification status: unverified
- Source: [arXiv:2608.01497](https://arxiv.org/abs/2608.01497)
- Category: architecture
- Solves: memory, stability, accuracy
- ML areas: transformer, training-dynamics, memory, initialization
- Math tags: algebra, combinatorics, linear-algebra, dynamical-systems
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a learned sequence-mixing matrix with a structured lower-triangular Sprugnoli operator whose square is exactly the identity. Applying the same operator in reverse reconstructs activations exactly, so it can be used as a reversible Transformer mixer or reversible channel permutation while retaining nontrivial long-range mixing.

## Mathematical statement

For power series g(x) with nonzero constant term and even f(x), the paper defines M(g,f) through the generating function M(g,f)(x,y)=g(x)/((1-x)(1+yx))+xf(x)/((1-x^2)(1-yx)). Its Sprugnoli representation is (G,F_1,x), where G(x)=g(x)/(1-x)+xf(x)/(1-x^2) and F_1(x)=x(xf(x)-(1+x)g(x))/(xf(x)+(1+x)g(x)). A Sprugnoli array (g_0,f_1,f_2) has entries T_{n,k}=[x^n]g_0(x)f_1(x)^{k mod 2}(xf_2(x))^{floor(k/2)}; here f_2(x)=x, so the even-column factor is x^2. Proposition 13 states that M(g,f)^2 is the aerated Appell operator with generating function a(x)=(g(x)g(-x)+x^2f(x)^2)/(1-x^2)^2. Therefore M(g,f) is an involution when a(x)=1, equivalently g(x)g(-x)=q(x):=(1-x^2)-x^2f(x)^2/(1-x^2). Since f is even, q is even. A constructive parameterization is g(x)=q(x)^(1/2) exp(h(x)), where h is any odd formal power series; then g(-x)=q(x)^(1/2) exp(-h(x)) and g(x)g(-x)=q(x) exactly.

## Key formulas

- $$M(g,f)(x,y)=\frac{g(x)}{(1-x)(1+yx)}+\frac{xf(x)}{(1-x^{2})(1-yx)}$$
- $$G(x)=\frac{g(x)}{1-x}+\frac{xf(x)}{1-x^{2}},\qquad F_1(x)=\frac{x\left(xf(x)-(1+x)g(x)\right)}{xf(x)+(1+x)g(x)},\qquad M(g,f)=(G,F_1,x)$$
- $$M(g,f)^2=\left(\frac{g(x)g(-x)+x^{2}f(x)^{2}}{(1-x^{2})^{2}},x\right)$$
- $$f(-x)=f(x),\quad q(x)=\left(1-x^{2}\right)-\frac{x^{2}f(x)^{2}}{1-x^{2}},\quad g(x)=q(x)^{1/2}e^{h(x)},\quad h(-x)=-h(x)\ \Longrightarrow\ M(g,f)^2=I$$

## Implementation notes

(1) Integration point: insert the mixer after each Transformer attention or MLP block, acting on the sequence axis for every feature channel, or on groups of d_model channels. For sequence length N, truncate g, f, G, and F_1 to degree N-1 and build the N by N lower-triangular matrix T with T[n,k] equal to the coefficient of x^n in G(x)F_1(x)^(k mod 2)x^(2 floor(k/2)). Use the same T in the reverse pass; because T^2=I analytically, the forward activation can be discarded and reconstructed as T(Tz). (2) Pseudocode: choose trainable even coefficients f(x)=1+sum_{j=1}^m f_{2j}x^(2j); compute q(x)=(1-x^2)-x^2f(x)^2/(1-x^2) by truncated polynomial arithmetic; compute s(x)=sqrt(q(x)) using s_0=sqrt(q_0) and s_n=(q_n-sum_{i=1}^{n-1}s_i s_{n-i})/(2s_0); choose trainable odd h(x)=sum_j h_{2j+1}x^(2j+1); compute g=s exp(h) with truncated convolution; compute G and F_1 from the displayed formulas, extract T, and set y=Tz. During backward, reconstruct z=T y before applying the block gradient. (3) Compute the power-series coefficients deterministically; estimate only numerical truncation error by measuring ||T^2-I||_F/N and reconstruction error ||T(Tz)-z||/||z||. Use stable polynomial recurrences rather than symbolic matrix squaring. (4) First experiment: train a 6-layer, 128-dimensional Transformer on WikiText-2 or Tiny Shakespeare with sequence length 128. Compare the proposed mixer against a learned dense lower-triangular mixer, a fixed reversal or permutation, and a standard reversible additive coupling layer at matched parameter count. Measure validation perplexity, peak activation memory, reconstruction error, gradient norm variance, and wall-clock throughput. The expected success signal is near-machine-precision reconstruction, substantially lower activation memory at equal depth, and no worse perplexity or loss descent than the learned triangular baseline. Also test longer sequences to determine whether coefficient extraction or dense T multiplication becomes the bottleneck.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.