# Cycle-Invariant Loss for Gauge-Free Matrix Prediction

- ID: 1570
- Canonical URL: https://synthcore.org/idea/1570/cycle-invariant-loss-for-gauge-free-matrix-prediction
- API JSON: https://synthcore.org/api/idea/1570.json
- API Markdown: https://synthcore.org/api/idea/1570.md
- Verification status: unverified
- Source: [arXiv:2607.29021](https://arxiv.org/abs/2607.29021)
- Category: geometry
- Solves: accuracy, stability, generalization
- ML areas: loss, graph-nn, world-model
- Math tags: linear-algebra, geometry, graph-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Train a neural network that predicts a symmetric matrix family without choosing a particular latent basis. In addition to matching pointwise eigenvalues, match gauge-invariant relational quantities formed by traces of products of matrices at several inputs; these distinguish matrix families that have identical spectra at every input but differ in their shared eigenvector geometry. Evaluate the result after one global orthogonal Procrustes alignment, not by independently aligning every sample.

## Mathematical statement

The paper defines a selected invariant vector \(\mathfrak{P}_{\rm sub}(\mathbf{x};H)\), which may contain spectral and loop-product components, and trains by \(\mathrm{RMSE}(\theta)=\left(|\mathcal{X}_{\rm train}|^{-1}\sum_{\mathbf{x}\in\mathcal{X}_{\rm train}}\|\mathfrak{P}_{\rm sub}(\mathbf{x};H_\theta)-\mathfrak{P}_{\rm sub}(\mathbf{x};H_\star)\|^2\right)^{1/2}\). Here \(H_\theta\) is the neural prediction, \(H_\star\) is the reference family, \(\mathcal{X}_{\rm train}\) is the training set, and \(\mathfrak{P}_{\rm sub}\) stacks the selected invariants. Recovery is evaluated modulo one constant orthogonal gauge using \(\mathrm{dist}(H_\theta,H_\star)=\min_{Q_0\in\mathsf{O}(N)}\max_{\mathbf{x}\in\mathcal{X}_{\rm test}}\|H_\theta(\mathbf{x})-Q_0^{\intercal}H_\star(\mathbf{x})Q_0\|_F\), where \(\mathsf{O}(N)\) is the orthogonal group and \(\|\cdot\|_F\) is the Frobenius norm. The adaptation exploits conjugation invariance: for orthogonal \(Q\), \(\operatorname{tr}((Q^{\intercal}A_1Q)\cdots(Q^{\intercal}A_kQ))=\operatorname{tr}(A_1\cdots A_k)\). For sampled inputs \(x_i\), define eigenvalue features \(s_i=\operatorname{eig}(H(x_i))\) and cycle features \(c_{i_1\ldots i_k}=\operatorname{tr}(H(x_{i_1})\cdots H(x_{i_k}))\). Use normalized cycle features \(\widetilde c_{i_1\ldots i_k}=c_{i_1\ldots i_k}/\prod_{j=1}^k(\|H(x_{i_j})\|_F+\epsilon)\) to avoid domination by matrix scale.

## Key formulas

- $$\mathrm{RMSE}(\theta)=\left(\frac{1}{|\mathcal{X}_{\rm train}|}\sum_{\mathbf{x}\in\mathcal{X}_{\rm train}}\bigl\|\mathfrak{P}_{\rm sub}(\mathbf{x};H_{\theta})-\mathfrak{P}_{\rm sub}(\mathbf{x};H_{\star})\bigr\|^{2}\right)^{1/2}.$$
- $$\mathrm{dist}(H_{\theta},H_{\star})=\min_{Q_{0}\in\mathsf{O}(N)}\max_{\mathbf{x}\in\mathcal{X}_{\rm test}}\bigl\|H_{\theta}(\mathbf{x})-Q_{0}^{\intercal}H_{\star}(\mathbf{x})Q_{0}\bigr\|_{F}.$$
- $$\operatorname{tr}\!\left((Q^{\intercal}A_1Q)(Q^{\intercal}A_2Q)\cdots(Q^{\intercal}A_kQ)\right)=\operatorname{tr}(A_1A_2\cdots A_k),\qquad Q^{\intercal}Q=I.$$
- $$\mathcal{L}=\frac{1}{B}\sum_{i=1}^{B}\|\operatorname{eig}(\widehat H_i)-\operatorname{eig}(H_i^\star)\|_2^2+\lambda\frac{1}{|\mathcal C|}\sum_{(i_1,\ldots,i_k)\in\mathcal C}\left(\widetilde c_{i_1\ldots i_k}(\widehat H)-\widetilde c_{i_1\ldots i_k}(H^\star)\right)^2.$$

## Implementation notes

Integrate this as an auxiliary loss for a network whose output is a real symmetric matrix \(\widehat H(x)\in\mathbb{R}^{N\times N}\), such as an orbital Hamiltonian model, latent covariance predictor, or matrix-valued world-model transition operator. Symmetrize the raw output with \(\widehat H=(A+A^{\mathsf T})/2\). For every minibatch, sample points \(x_1,\ldots,x_B\) and construct a cycle set \(\mathcal C\): include pairs \((i,j)\), random ordered triples \((i,j,k)\), or short cycles of a k-nearest-neighbor graph in input space. Compute eigenvalues using a differentiable symmetric eigensolver. For every cycle, compute \(c_{i_1\ldots i_k}=\operatorname{tr}(\widehat H_{i_1}\cdots\widehat H_{i_k})\), then normalize it by \(\prod_j(\|\widehat H_{i_j}\|_F+10^{-6})\). Cache the corresponding reference invariants from \(H_\star\), and optimize the displayed loss with spectral weight one and \(\lambda\in\{0.01,0.1,1\}\). Begin with pair products and add triples if optimization remains well-conditioned. The conjugation identities are exact; normalization, cycle sampling, and loss weights are empirical choices. For evaluation, concatenate matrices over test inputs, compute one orthogonal Procrustes matrix \(Q_0\) by SVD, and report aligned Frobenius error, eigenvalue error, and unaligned error. First test a two- or four-channel MLP on synthetic families \(H_\star(x)=Q_\star^{\mathsf T}(A_0+\sum_m x_mA_m+\sin(x_1)B)Q_\star\), comparing spectrum-only, entrywise MSE, and spectrum-plus-cycle training. Success is lower aligned test error, robustness to replacing labels by a different global orthogonal conjugation, and better extrapolation at equal FLOPs.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.