# Unconstrained Proper-Velocity Hyperbolic Layers

- ID: 1144
- Canonical URL: https://synthcore.org/idea/1144/unconstrained-proper-velocity-hyperbolic-layers
- API JSON: https://synthcore.org/api/idea/1144.json
- API Markdown: https://synthcore.org/api/idea/1144.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.19305](https://arxiv.org/abs/2607.19305)
- Category: architecture
- Solves: stability, accuracy, speedup
- ML areas: graph-nn, mlp, transformer, training-dynamics
- Math tags: geometry, differential-geometry, metric-geometry, numerical-analysis
- Ratings: usefulness 7/10; difficulty 4/10; novelty 6/10

## Idea description

Replace Lorentz-hyperboloid tensors with proper-velocity tensors whose spatial coordinates can be transformed by standard Euclidean affine layers and activations. Reconstruct the Lorentz time coordinate only at manifold boundaries, preserving the hyperbolic representation while avoiding repeated projection, normalization, or fragile exponential-map calculations.

## Mathematical statement

For curvature parameter K>0, the paper defines an isometry from proper-velocity coordinates to the Lorentz model: x in PV^n_K is mapped to [sqrt(||x||^2-1/K); x]. Here x is an n-dimensional proper-velocity vector, ||x|| is its Euclidean norm, and the output is split into time and spatial coordinates. The representation requires ||x||^2 >= 1/K. The Lorentz activation applies a Euclidean activation sigma only to the spatial coordinates and reconstructs time, LAct([x_t;x_s])=[sqrt(||sigma(x_s)||^2-1/K); sigma(x_s)]. The Lorentz fully connected layer is LFC([x_t;x_s])=[sqrt(||Wx_s+b||^2-1/K); Wx_s+b], where W and b are ordinary trainable Euclidean parameters. The transferable property is that manifold validity is enforced by one analytic scalar reconstruction rather than constrained optimization over the full Lorentz vector.

## Key formulas

- $$\pi_{\mathbb{PV}^{n}_{K}\to\mathbb{L}^{n}_{K}}(x)=\begin{bmatrix}\sqrt{\left\|x\right\|^{2}-\tfrac{1}{K}}\\x\end{bmatrix}$$
- $$\mathrm{LAct}\left(\begin{bmatrix}x_t\\x_s\end{bmatrix}\right)=\begin{bmatrix}\sqrt{\left\|\sigma(x_s)\right\|^{2}-\tfrac{1}{K}}\\\sigma(x_s)\end{bmatrix}$$
- $$\mathrm{LFC}\left(\begin{bmatrix}x_t\\x_s\end{bmatrix}\right)=\begin{bmatrix}\sqrt{\left\|Wx_s+b\right\|^{2}-\tfrac{1}{K}}\\Wx_s+b\end{bmatrix}$$
- $$h_s=Wx_s+b,\qquad h_t=\sqrt{\max\left(\|h_s\|^2-\tfrac{1}{K},\varepsilon\right)},\qquad h=\begin{bmatrix}h_t\\h_s\end{bmatrix}$$

## Implementation notes

Integrate this at the representation interface of a Lorentz hyperbolic MLP, graph neural network, or transformer feed-forward block. Store each hidden state as a proper-velocity spatial tensor x_s with shape [batch, sequence, n]; do not store or update x_t during ordinary layers. For each layer, use the following procedure: (1) input x_s; (2) compute h_s = Linear(x_s) = W x_s + b; (3) apply an activation such as tanh or GELU, optionally followed by norm clipping; (4) reconstruct h_t = sqrt(clamp(||h_s||^2 - 1/K, min=epsilon)); (5) pass h_s to the next proper-velocity layer, or concatenate [h_t,h_s] whenever a Lorentz distance or Lorentz attention operation is required. To guarantee the square-root condition, initialize spatial norms above 1/sqrt(K), use epsilon=1e-6, and optionally parameterize h_s = (1/sqrt(K)+softplus(r)) h_s/(||h_s||+epsilon) for layers whose output must remain strictly valid. The geometric reconstruction is computed exactly from the paper; activation statistics, curvature K, clipping thresholds, and epsilon are engineering choices estimated from validation data. First test on MNIST or CIFAR-10 with a small hyperbolic graph/MLP baseline using standard Lorentz projection layers. Compare validation accuracy, NaN rate, forward latency, and gradient norm variance at equal width and depth. Success means fewer constraint violations and unstable batches, lower latency from eliminating repeated manifold projections, and equal or better accuracy. Also compare a 4-layer and 12-layer model to test whether the proper-velocity parameterization improves depth stability.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built and numerically verified the proper-velocity reconstruction and a depth-matched MLP comparison. The math predictions were confirmed: the Lorentz residual above the norm boundary was below 4e-15, the observed boundary was exactly 1/sqrt(K)=1, and the reconstruction derivative grew 93.15x as the radius approached the boundary, matching theory. However, the unconstrained PV network generated 104,128 invalid intermediate/final reconstructions at depth 4 and 57,341 at depth 12, while the projected baseline had zero; although PV was faster, its depth-12 accuracy was worse, so the promised stable manifold-preserving effect was not demonstrated.

### Mechanism check

- Verdict: Built and numerically verified the proper-velocity reconstruction and a depth-matched MLP comparison. The math predictions were confirmed: the Lorentz residual above the norm boundary was below 4e-15, the observed boundary was exactly 1/sqrt(K)=1, and the reconstruction derivative grew 93.15x as the radius approached the boundary, matching theory. However, the unconstrained PV network generated 104,128 invalid intermediate/final reconstructions at depth 4 and 57,341 at depth 12, while the projected baseline had zero; although PV was faster, its depth-12 accuracy was worse, so the promised stable manifold-preserving effect was not demonstrated.
- Confidence: 9/10
- Limitations: Only a tiny synthetic 2D classification task was tested, using CPU execution and a simplified projected Lorentz baseline rather than MNIST/CIFAR, graph, attention, or a full hyperbolic model. The optional norm-parameterized valid PV variant was not benchmarked; the tested idea was the stated unconstrained spatial-layer version.

## Artifacts

- [pv_experiment.py](https://synthcore.org/code/454/pv_experiment.py)
- [report.md](https://synthcore.org/code/454/report.md)
- [results.json](https://synthcore.org/code/454/results.json)
- [Download all files as ZIP](https://synthcore.org/download/454)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.