# Hurwitz–Radon signed bilinear mixer - ID: 1980 - Canonical URL: https://synthcore.org/idea/1980/hurwitz-radon-signed-bilinear-mixer - API JSON: https://synthcore.org/api/idea/1980.json - API Markdown: https://synthcore.org/api/idea/1980.md - Verification status: unverified - Source: [arXiv:2608.08246](https://arxiv.org/abs/2608.08246) - Category: architecture - Solves: stability, memory, accuracy - ML areas: mlp, initialization, training - Math tags: linear-algebra, algebra, random-matrix - Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10 ## Idea description Replace a learned dense bilinear map with a structured family of signed orthogonal matrices. Given feature vectors y,z in R^n, produce r interaction features h_a = y^T H_a z / sqrt(n), where the H_a form a Hadamard/Clifford-like family; the resulting bilinear map has operator norm at most one when r is within the Hurwitz–Radon limit. Learn only channel projections and optional scalar gates around this fixed mixer. ## Mathematical statement The paper studies trilinear sign forms A(x,y,z)=sum_{a=1}^r sum_{i,j=1}^n epsilon_{aij} x_a y_i z_j on R^r x R^n x R^n, with epsilon_{aij} in {-1,1}. It proves every such form has norm at least sqrt(n), and equality is possible exactly when a Hadamard matrix of order n exists and r <= rho(n), where rho is the Hurwitz–Radon function. Use matrices H_a in R^{n x n} with entries in {-1,1} satisfying H_a^T H_a=n I_n and H_a^T H_b + H_b^T H_a=0 for a != b. Define B(y,z) in R^r by B_a(y,z)=y^T H_a z/sqrt(n). For x in R^r, A(x,y,z)=sum_a x_a y^T H_a z/sqrt(n)=x^T B(y,z). The equality construction implies ||A||=1, hence ||B(y,z)||_2 <= ||y||_2 ||z||_2. The general KSZ comparison bound is theta(p_1,...,p_d)=1/min_j max{2,p_j^*}, alpha(p_j)=max{1/2-1/p_j,0}, with a sign form satisfying ||A|| <= C_d (sum_j n_j)^{theta} product_j n_j^{alpha(p_j)}. Here p_j^* is the conjugate exponent, n_j is the dimension of input j, and C_d depends only on d. ## Key formulas - $$A(z^{(1)},\ldots,z^{(d)})=\sum_{j_{1},\ldots,j_{d}}\varepsilon_{j_{1}\cdots j_{d}}z_{j_{1}}^{(1)}\cdots z_{j_{d}}^{(d)},\qquad \varepsilon_{j_{1}\cdots j_{d}}\in\{-1,1\}.$$ - $$\vartheta(p_{1},\ldots,p_{d}):=\frac{1}{\min_{1\leq j\leq d}\max\{2,p_{j}^{*}\}},\qquad\alpha(p_{j}):=\max\left\{\frac{1}{2}-\frac{1}{p_{j}},0\right\}.$$ - $$\|A\|\leq C_{d}\left(\sum_{j=1}^{d}n_{j}\right)^{\vartheta(p_{1},\ldots,p_{d})}\prod_{j=1}^{d}n_{j}^{\alpha(p_{j})}.$$ - $$H_a^T H_a=nI_n,\quad H_a^T H_b+H_b^T H_a=0\ (a\neq b),\quad B_a(y,z)=\frac{y^T H_a z}{\sqrt n},\quad \|B(y,z)\|_2\leq\|y\|_2\|z\|_2.$$ ## Implementation notes Integrate this as a multiplicative feature block between two ordinary linear projections. For an input token or image feature u, compute y=P_y u and z=P_z u, each in R^n, then form h in R^r with the fixed signed matrices H_a. The forward pass is h_a = (y^T H_a z)/sqrt(n); apply LayerNorm or a learned scalar gate g_a only after this operation, followed by an output projection P_o. Use n=4, 8, 16, or 32 initially and choose r no larger than the available Hurwitz–Radon family size. For an MVP, use a Hadamard matrix and generate or search signed anticommuting companions offline, storing H_a as int8 tensors. If an exact family is unavailable, initialize trainable H_a with signed matrices and penalize violations using L_struct = sum_a ||H_a^T H_a/n-I||_F^2 + sum_{a