✓ Mechanism works 2026

Lorentz-Gram-preserving hyperbolic attention

Implementation & benchmark of arXiv:2608.22568 — Distance preservers for Lobachevsky space

Usefulness7/10

Difficulty5/10

Novelty7/10

Geometry Accuracy Stability Generalization Attention Graph nn Embedding Geometry Metric geometry Convex analysis Probability

Source paper: Distance preservers for Lobachevsky space arXiv:2608.22568 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained nonlinearity on hyperbolic pairwise similarities with a function from the paper's exact Lorentz–Gram preserver family. The transformed similarity matrix remains realizable as Lorentz inner products of future-directed unit timelike vectors, allowing a network to sharpen or smooth hyperbolic neighborhoods without introducing geometrically impossible pairwise relations.

Formulas

$$f(t)=1+\gamma 1_{(1,\infty)}(t)+\beta\frac{t^{\alpha}-1}{\alpha}+\int_{(0,\infty)}(1-t^{-s})\,\mathrm{d}\nu(s),\qquad t\ge1,$$

$$\frac{t^{\alpha}-1}{\alpha}=\log t\quad\text{if }\alpha=0,$$

$$\beta\geq\alpha\bigl(1+\gamma+\nu((0,\infty))\bigr)\quad(\alpha\in(0,1]),\qquad \int_{(0,\infty)}\min\{s,1\}\,\mathrm{d}\nu(s)<\infty\quad(\alpha=0),$$

$$\widetilde G_{ij}=f(G_{ij}),\qquad G_{ij}=\langle x_i,x_j\rangle_L=x_{i0}x_{j0}-\sum_{k=1}^{d}x_{ik}x_{jk}.$$

Mathematical statement

For a future-directed unit timelike vector x=(x_0,x_1,...,x_d) with Lorentz product $\langle x,y\rangle_L=x_0y_0-\sum_{k=1}^d x_ky_k$, one has $\langle x_i,x_j\rangle_L\ge1$. A Lorentz–Gram matrix is $G_{ij}=\langle x_i,x_j\rangle_L$, with diagonal one and off-diagonal entries at least one, together with the appropriate Lorentz signature. The paper states that entrywise application preserves the class of Lorentz–Gram matrices exactly when $f(t)=1+\gamma 1_{(1,\infty)}(t)+\beta(t^\alpha-1)/\alpha+\int_{(0,\infty)}(1-t^{-s})\,d\nu(s)$, where $\alpha\in[0,1]$, $\beta,\gamma\ge0$, and $\nu$ is a positive Radon measure. At $\alpha=0$, the power term is $\log t$. For $\alpha>0$, $\beta\ge\alpha(1+\gamma+\nu((0,\infty)))$; for $\alpha=0$, $\int\min\{s,1\}\,d\nu(s)<\infty$. Use a finite positive mixture $\nu=\sum_{m=1}^M q_m\delta_{s_m}$, with $q_m,s_m\ge0$, as the trainable implementation.

Implementation notes

Integrate this as a hyperbolic similarity layer before attention or graph aggregation. Inputs are a batch of Lorentz points X with shape B×N×(d+1), normalized so every row satisfies x_0^2-||x_{1:d}||_2^2=1 and x_0>0. First compute G=X[:,:,0]X[:,:,0]^T-X[:,:,1:]X[:,:,1:]^T. Clamp off-diagonal values below 1 to 1+10^-6 for numerical stability and set the diagonal to one. Parameterize alpha=sigmoid(a) in (0,1), s_m=softplus(u_m), and q_m=softplus(v_m). With gamma=0, set beta=alpha*(1+sum_m q_m+epsilon), which enforces the theorem's sufficient constraint. Evaluate f(G)=1+beta*expm1(alpha*log(G))/alpha+sum_m q_m*(1-exp(-s_m*log(G))); use beta*log(G) when alpha is near zero. Subtract the rowwise maximum from f(G) and apply softmax to obtain attention weights, or pass f(G) directly as a message-passing kernel. If explicit coordinates are required, factorize the Lorentz matrix using the eigendecomposition of J*f(G), where J=diag(1,-1,...,-1), retaining one positive and the required negative eigenvalues. The mathematical guarantee comes from the preserver theorem; empirically monitor diagonal error, minimum off-diagonal value, and the number of incorrect-signature eigenvalues. Start with a two-dimensional synthetic hyperbolic classification problem and Cora link prediction using a hyperbolic GNN. Compare Euclidean attention, raw hyperbolic attention, and the constrained transform at equal hidden dimension and FLOPs. Success is higher validation accuracy or MRR, fewer invalid factorizations, and lower seed-to-seed variance.

Verification

Mechanism works

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a trainable finite-mixture Lorentz–Gram-preserving attention layer with positive softplus mixture parameters and theorem-constrained beta. Across 12 numerical trials, the constrained transform preserved one-positive Lorentz signature in 100% of cases, while an unconstrained exponential transform did so in 0%; the trainable layer also passed a differentiability and shape smoke test. In the five-seed synthetic classification experiment, constrained attention achieved 1.000 accuracy with zero seed variance, matching Euclidean attention and improving over raw hyperbolic attention at 0.971 accuracy with 0.028 standard deviation.

Agent confidence: 8/10

Baseline: Euclidean attention: validation accuracy 1.000 ± 0.000; raw hyperbolic attention: 0.971 ± 0.028.
Idea: Constrained attention: validation accuracy 1.000 ± 0.000, mean train loss 0.0455; signature preservation 12/12, diagonal error 0, minimum transformed off-diagonal 1.000019.

Limitations: Only a small synthetic node-classification task was tested; no Cora link prediction, explicit coordinate factorization, GPU benchmark, FLOP/speed comparison, or end-to-end training of the attention parameters was evaluated.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 3 files · code, reports and structured results

🐍 experiment.py 6.8 KB View Raw
📄 report.md 1.5 KB View
📊 results.json 2.4 KB View Raw