Inverse-Eigenvector Tight-Frame Codebook

(1) Exact integration point: use the construction as an offline-generated codebook for an embedding layer, an MoE expert/router direction matrix, or the key and value direction matrices of a small attention head. Let the feature width be n and choose N at least n. Estimate M as the empirical correlation matrix of normalized calibration embeddings, then regularize it with M ← (1 − ε)M + εI to avoid singularity. Store the αᵢ vectors and normalized weights cᵢ as an initialization; optionally allow a small trainable correction to each αᵢ. (2) Pseudocode: normalize calibration samples h; compute M; initialize every αᵢ with nonzero coordinates; repeat, gᵢ = Mαᵢ − αᵢ⁻¹, update αᵢ ← αᵢ − η times a damped Adam or Newton step on gᵢ, and clip |αᵢⱼ| below by δ; then optimize unconstrained logits uᵢ with c = softmax(u) using the frame loss. Alternate root-residual and frame-residual updates. For a router, compute logits ℓᵢ = hᵀαᵢ divided by square root of n, then apply softmax or top-k routing and add β times the frame loss to the neural training objective. (3) Computed from the mathematics: inverse-eigenvector residuals, simplex-constrained positive weights, and the Frobenius tight-frame residual. Estimated empirically: M, ε, damping, clipping δ, and the regularization coefficient β. (4) First experiment: train a width-64 MLP-Mixer or four-expert top-2 MoE on CIFAR-10, comparing this codebook with Gaussian and orthogonal initialization. Measure loss versus optimizer steps, feature-coordinate gradient variance, frame residual, router expert-load entropy, and final accuracy. Success means faster early loss descent at equal FLOPs, lower gradient anisotropy, and fewer collapsed MoE experts without sacrificing final accuracy.