# Jordan spectral feature scaling

- ID: 2283
- Canonical URL: https://synthcore.org/idea/2283/jordan-spectral-feature-scaling
- API JSON: https://synthcore.org/api/idea/2283.json
- API Markdown: https://synthcore.org/api/idea/2283.md
- Verification status: unverified
- Source: [arXiv:2608.14231](https://arxiv.org/abs/2608.14231)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: mlp, transformer, architecture
- Math tags: geometry, convex-analysis, algebra, functional-analysis
- Ratings: usefulness 5/10; difficulty 5/10; novelty 8/10

## Idea description

Group neural features into small Hermitian matrix elements and scale each group with the paper's tracial spectral Lp norm rather than independently normalizing scalar channels. This introduces a coupled spectral geometry while remaining implementable with ordinary eigendecompositions in the associative Hermitian-matrix special case.

## Mathematical statement

For a JBW*-algebra \(\mathcal M\) with involution \(x\mapsto x^*\), Jordan product \(x\circ y\), and normal finite faithful trace \(\tau\), the paper defines \(\|x\|_{L^p(\mathcal M)}=\bigl(\tau[(x^*\circ x)^{p/2}]\bigr)^{1/p}\) for \(1\le p<\infty\). The element \(x^*\circ x\) is positive, its power is defined by Jordan spectral calculus, and the main theorem proves that this quantity is a genuine norm, including the triangle inequality in the nonassociative setting. For implementation use \(\mathcal M=H_d(\mathbb C)\), the Hermitian \(d\times d\) matrices, with \(A\circ B=(AB+BA)/2\) and normalized trace \(\tau(A)=\operatorname{tr}(A)/d\). If \(X\) is Hermitian, then \(X^*\circ X=X^2\), so \(\|X\|_{p,\tau}=(d^{-1}\operatorname{tr}(|X|^p))^{1/p}\), where \(|X|=(X^2)^{1/2}\). The adapted neural operation is \(\widehat X=X/(\|X\|_{p,\tau}+\varepsilon)\). The paper also establishes a Jordan analogue of joint convexity for the Kiefer map \(\Phi(a,h)=a^*\circ h^{-1}\circ a\), where \(h\) is positive definite; this supports optional convex inverse-energy penalties but is not required for the first experiment.

## Key formulas

- $$\|x\|_{\mathrm{L}^{p}(\mathcal{M})}\overset{\mathrm{def}}{=}\left(\tau\left[(x^{*}\circ x)^{\frac{p}{2}}\right]\right)^{\frac{1}{p}},\qquad 1\leqslant p<\infty.$$
- $$A\circ B=\frac{AB+BA}{2},\qquad \tau(A)=\frac{1}{d}\operatorname{tr}(A),\qquad \|X\|_{p,\tau}=\left(\frac{1}{d}\operatorname{tr}(|X|^p)\right)^{1/p}.$$
- $$\widehat X=\frac{X}{\left(\tau\left[(X^{*}\circ X)^{p/2}\right]\right)^{1/p}+\varepsilon},\qquad \varepsilon>0.$$
- $$\Phi(a,h)=a^{*}\circ h^{-1}\circ a,\qquad \Phi(\theta a_1+(1-\theta)a_2,\theta h_1+(1-\theta)h_2)\preceq_J\theta\Phi(a_1,h_1)+(1-\theta)\Phi(a_2,h_2),\quad 0\leq\theta\leq1.$$

## Implementation notes

Integrate the operation after an MLP block, attention projection, or residual branch. Let the feature width be partitioned into groups of size equal to the real coordinate count of a Hermitian matrix; for a simple implementation use real symmetric \(d\times d\) matrices, requiring \(d(d+1)/2\) coordinates per group. Given \(z\in\mathbb R^{B\times T\times Gd(d+1)/2}\), reshape each group into a symmetric matrix \(X\), compute its eigendecomposition \(X=U\operatorname{diag}(\lambda_1,\ldots,\lambda_d)U^\top\), and calculate \(n=(d^{-1}\sum_i(|\lambda_i|+\delta)^p)^{1/p}\). Return \(y=\gamma X/(n+\varepsilon)+\beta\), flattening the matrices back to channels before the next layer or residual addition. Here \(p\in\{2,4\}\), \(\gamma\) and \(\beta\) are learned per channel or group, \(\delta>0\) smooths the absolute value, and \(\varepsilon>0\) prevents division by zero. The eigendecomposition implements the paper's spectral functional calculus; no theorem constant needs to be estimated. Use stop-gradient through \(n\) as an ablation, but the primary version should backpropagate through the eigendecomposition. Begin with \(d=2\) or \(3\) in a 6-layer Transformer on CIFAR-10 patches or a small character-level language model, comparing against LayerNorm, RMSNorm, and GroupNorm at equal width and parameter count. Record loss curves at equal FLOPs, gradient-norm spikes, activation norms, validation accuracy or perplexity, and wall-clock overhead. The first falsifiable success criterion is comparable throughput with either fewer unstable runs, faster loss descent, or better validation performance under a fixed training budget. Also test robustness by evaluating corrupted inputs or modest feature-scale perturbations; the spectral norm should preserve performance better if its coupled geometry is useful.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.