# Bilinear-Form Structured Transition

- ID: 721
- Canonical URL: https://synthcore.org/idea/721/bilinear-form-structured-transition
- API JSON: https://synthcore.org/api/idea/721.json
- API Markdown: https://synthcore.org/api/idea/721.md
- Verification status: unverified
- Source: [arXiv:2607.10956](https://arxiv.org/abs/2607.10956)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: ssm, rnn, initialization, training-dynamics
- Math tags: linear-algebra, symplectic, geometry, representation-theory, dynamical-systems
- Ratings: usefulness 5/10; difficulty 5/10; novelty 5/10

## Idea description

Construct the latent transition from a nondegenerate bilinear form phi and a form-compatible operator instead of from an unconstrained dense matrix. The resulting SSM has an exact orthogonal or symplectic algebraic structure, reducing transition parameter redundancy and testing whether preservation of a latent pairing improves extrapolation on reversible, parity-sensitive, or Hamiltonian-like sequence tasks.

## Mathematical statement

The paper uses an isomorphism phi:A to A^vee defining a nondegenerate orthogonal or symplectic pairing <v,w>, and imposes alpha^vee phi=phi alpha. In coordinates this is <v,alpha w>=<alpha v,w>, so alpha is self-adjoint with respect to the pairing. The extracted parameter-space identification is V_+=Sym^2 A^vee direct-sum Hom(A,U) in the symmetric case and V_-=Lambda^2 A^vee direct-sum Hom(A,U) in the skew case. Here A is the latent state space, U is the feature or output space, phi is invertible, alpha^vee is the transpose/dual map, and Sym^2 or Lambda^2 selects symmetric or skew-symmetric matrices. The transferable construction is exact parameter tying: choose a free symmetric matrix S or skew-symmetric matrix K and set alpha=phi^{-1}S or alpha=phi^{-1}K. This guarantees alpha^T phi=phi alpha by construction. For a symplectic pairing use even d and the canonical matrix J=[[0,I],[-I,0]].

## Key formulas

- $$\alpha^{\vee}\phi=\phi\alpha.$$
- $$\left\langle v,\alpha w\right\rangle=\left\langle\alpha v,w\right\rangle.$$
- $$V_{+}=\operatorname{Sym}^{2}A^{\vee}\oplus\operatorname{Hom}(A,U),\qquad V_{-}=\Lambda^{2}A^{\vee}\oplus\operatorname{Hom}(A,U).$$
- $$\alpha=\phi^{-1}S\quad(S^{\mathsf T}=S),\qquad\text{or}\qquad \alpha=\phi^{-1}K\quad(K^{\mathsf T}=-K).$$

## Implementation notes

(1) Integration point: replace the transition matrix in an SSM or recurrent block. Let h_t be d-dimensional, with even d for the skew case. Fix phi=I for the symmetric pairing, or phi=J=[[0,I],[-I,0]] for a canonical symplectic pairing. Learn an unconstrained matrix R and derive S=(R+R^T)/2 or K=(R-R^T)/2, then set alpha=solve(phi,S) or alpha=solve(phi,K). Use h_{t+1}=rho(alpha h_t+B u_t), with ordinary learned input B and readout C. Initially remove the recurrent nonlinearity or put it only in a residual branch so the algebraic effect can be isolated. (2) Pseudocode: R is a trainable d-by-d matrix; if symmetric, S=0.5*(R+R.T), alpha=linear_solve(phi,S); if skew, K=0.5*(R-R.T), alpha=linear_solve(phi,K); optionally rescale alpha by c/max(1,rho(alpha)) to control the discrete-time spectral radius; then run the recurrence. (3) Computed exactly from the paper’s math: symmetry or skew-symmetry of the free parameter and the identity alpha^T phi=phi alpha. Estimated empirically: spectral radius, long-horizon gradient norm, pairing error ||alpha^T phi-phi alpha||_F, and task error. Compare fixed phi against a learnable invertible phi parameterized by a matrix exponential. (4) First experiment: compare dense, orthogonal, symmetric-pairing, and symplectic-pairing SSMs with width 64 on Copy, reverse-copy, parity, and synthetic Hamiltonian trajectory prediction. Match parameter counts and training FLOPs. Measure validation accuracy, long-horizon prediction error, gradient norm, spectral radius, pairing error, and wall-clock throughput. Success is lower extrapolation error or better reversal/parity accuracy at equal parameters, with pairing error near machine precision and no substantial per-step slowdown.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.