Near-Parseval Unitary Orbit Memory

(1) Integration point: implement a compact associative-memory layer before an attention readout, or use it as the key/value generator in a small GRU or transformer. Let the seed $\phi\in\mathbb R^d$ be learned and define a differentiable orthogonal action $\pi_\theta(\gamma)$. For an MVP, use several learned skew-symmetric matrices $A_\ell$ and candidate group parameters $(t,\ell)$ with $\pi_\theta(t,\ell)=\exp(tA_\ell)$. Approximate the exponential with the Cayley transform $(I-tA_\ell/2)^{-1}(I+tA_\ell/2)$ to preserve norms. (2) Pseudocode: set $V$ empty; for each candidate $(t,\ell)$, compute $v=\pi_\theta(t,\ell)\phi$ and normalize it; compute $c=V^*v$ and $r=v-V(V^*V+\lambda I)^{-1}c$; reject if $\|r\|_2^2<\tau$ or $\max_i|c_i|>\mu$; otherwise append $v$ to $V$. For a query $q$, compute attention weights $a=\operatorname{softmax}(\beta V^*q)$ and return $Va$, or use the ridge frame reconstruction $V(V^*V+\lambda I)^{-1}V^*q$. (3) Compute from the mathematics the unitary orbit, residual-based sequential selection, Gram matrix, and spectral frame bounds. Estimate $\lambda_{\min}(G)$ and $\lambda_{\max}(G)$ by eigendecomposition or power iteration, and detect projective duplicates using $|v_i^*v_j|\approx1$. Add $L_{\mathrm{frame}}=\|G-I\|_F^2$ if hard selection is too restrictive. (4) First experiment: use a one-layer GRU or small transformer on Sequential MNIST and a synthetic key-value recall task. Compare against independently learned slots and standard cosine attention with equal dimension and slot count. Measure recall accuracy, Gram condition number, gradient-norm variance, trainable parameter count, and latency. The method succeeds if it maintains eigenvalues near $1$, reduces long-context recall variance, and matches baseline accuracy with fewer stored parameters. Include ablations removing unitary transforms, sequential rejection, and frame regularization.