Hurwitz–Radon signed bilinear mixer

Integrate this as a multiplicative feature block between two ordinary linear projections. For an input token or image feature u, compute y=P_y u and z=P_z u, each in R^n, then form h in R^r with the fixed signed matrices H_a. The forward pass is h_a = (y^T H_a z)/sqrt(n); apply LayerNorm or a learned scalar gate g_a only after this operation, followed by an output projection P_o. Use n=4, 8, 16, or 32 initially and choose r no larger than the available Hurwitz–Radon family size. For an MVP, use a Hadamard matrix and generate or search signed anticommuting companions offline, storing H_a as int8 tensors. If an exact family is unavailable, initialize trainable H_a with signed matrices and penalize violations using L_struct = sum_a ||H_a^T H_a/n-I||_F^2 + sum_{a<b} ||(H_a^T H_b+H_b^T H_a)/n||_F^2. Pseudocode: y=P_y(u); z=P_z(u); for a in 1..r, h[a]=dot(y,H[a] @ z)/sqrt(n); v=g*h; out=P_o(v); loss=task_loss(out)+lambda*L_struct if matrices are trainable. The exact mathematical quantity is the normalized operator bound ||B||=1. Empirically estimate it by maximizing ||B(y,z)|| over random unit vectors and by 20-step alternating power iteration. First test on a 4-layer CIFAR-10 MLP or a small bilinear classifier, comparing this block against a learned dense bilinear tensor with matched parameter count, a random-sign tensor, and a standard MLP. Track gradient norm variance, activation growth with depth, validation accuracy, and wall-clock/FLOPs. Success is stable training without spectral normalization, lower gradient explosion, and equal or better accuracy at substantially fewer learned parameters; a useful target is at least 10x fewer bilinear parameters with no more than 1 percentage point accuracy loss.