Unverified 2026

Indefinite Grassmann Adapter

Usefulness5/10

Difficulty5/10

Novelty8/10

Architecture Accuracy Stability Memory Embedding Attention Fine tuning Geometry Linear algebra Lie groups

Source paper: The Grassmannian of indefinite subspaces arXiv:2608.30249 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained low-rank adapter or similarity projection with a learned subspace carrying a prescribed signed metric. The module learns an orthonormal basis U for a k=p+q dimensional subspace, forces the compressed form U^*I_{m,n}U to have p positive and q negative eigenvalues, and uses the resulting pseudo-inner product for signed attention or retrieval scores.

Formulas

$$\Iso_d(\mathbb{F})=\{Q\in\mathbb{F}^{d\times d}:Q^*Q=I\},\qquad P_{\mathbb{W}}=UU^*,\qquad U^*U=I_k.$$

$$I_{m,n}=\operatorname{diag}(I_m,-I_n),\qquad B=U^*I_{m,n}U.$$

$$\mathbb{W}\in\operatorname{Gr}_{p,q}(\mathbb{F}^{m+n})\quad\Longleftrightarrow\quad \mathsf{s}(P_{\mathbb{W}}I_{m,n}P_{\mathbb{W}})=(p,q,d-k),\qquad k=p+q.$$

$$\operatorname{score}_U(x,y)=x^*U(B+\varepsilon I_k)^{-1}U^*y,\qquad \mathcal{L}_{\mathrm{sig}}=\sum_{i=1}^{p}[\max(0,\delta-\lambda_i(B))]^2+\sum_{i=p+1}^{k}[\max(0,\delta+\lambda_i(B))]^2.$$

Mathematical statement

Let I_{m,n}=\operatorname{diag}(I_m,-I_n) be the fixed nondegenerate bilinear-form matrix on \mathbb{F}^{m+n}, where \mathbb{F}\in\{\mathbb{R},\mathbb{C},\mathbb{H}\}. Let U\in\mathbb{F}^{(m+n) imes k} have Euclidean-orthonormal columns, U^*U=I_k, and let \mathbb{W}=\operatorname{col}(U). The paper characterizes the indefinite Grassmannian by the signature of the compressed form B=U^*I_{m,n}U: \mathbb{W}\in\operatorname{Gr}_{p,q}(\mathbb{F}^{m+n}) if and only if \mathsf{s}(P_{\mathbb{W}}I_{m,n}P_{\mathbb{W}})=(p,q,d-k), equivalently B has p positive and q negative eigenvalues on the k-dimensional subspace, with k=p+q. Here P_{\mathbb{W}}=UU^* is the Euclidean orthogonal projector and \mathsf{s} counts positive, negative, and zero eigenvalues. The implementation uses the nondegenerate compressed matrix B to define a signed latent metric and a margin penalty that keeps its eigenvalues away from zero with the desired signs.

Implementation notes

Integrate this as a replacement for a low-rank similarity head, a LoRA-style adapter, or an attention head's query-key metric. Choose ambient feature width d=m+n and latent width k=p+q, with p positive and q negative coordinates. Store an unconstrained parameter A in R^{d x k}; at every forward pass compute a differentiable thin QR factorization A=UR, retaining U=Q[:, :k]. Form I=diag(+1 repeated m times, -1 repeated n times) and compute B=U^T I U. For query and key vectors x and y, compute z_x=U^T x and z_y=U^T y, then use score=z_x^T(B+epsilon I_k)^{-1}z_y. In attention, divide this score by the normal temperature before softmax. The training loop is: compute U by QR; compute B; eigendecompose B=V diag(lambda) V^T; evaluate task loss; add lambda_sig times L_sig; backpropagate into A. Sort eigenvalues descending, require the first p to exceed margin delta and the final q to be below -delta. The paper contributes the projector and signature characterization; QR, inverse regularization, eigenvalue margins, and the attention integration are the adaptation. Compute B and its eigenvalues exactly per minibatch, while monitoring the minimum absolute eigenvalue as a conditioning diagnostic. First test a small Transformer classifier on CIFAR-10 patch tokens or AG News, comparing ordinary LoRA, an unconstrained low-rank metric, and this adapter at equal rank and parameter count. Measure validation accuracy, training loss at equal FLOPs, gradient norms, logit variance, and eigenvalue-sign violations. Success means higher accuracy or retrieval separation at equal parameter count without eigenvalue collapse or attention-logit explosion.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.