△ Mechanism confirmed, baseline not beaten 2026

Cholesky-Structured SPD Classifier

Implementation & benchmark of arXiv:2607.19305 — Riemannian Deep Learning: Modules, Networks, and Geometries

Usefulness7/10

Difficulty6/10

Novelty7/10

Architecture Speedup Stability Accuracy Graph nn Cnn Mlp Training dynamics Inference speedup Geometry Linear algebra Matrix analysis Numerical analysis

Source paper: Riemannian Deep Learning: Modules, Networks, and Geometries arXiv:2607.19305 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build an SPD classifier and residual head directly from Cholesky factors, using lower-triangular differences and matrix-power terms instead of generic eigendecomposition-based logarithm operators. This retains covariance geometry while making positive-definiteness automatic and backpropagation more numerically stable for minibatch training.

Formulas

$$S=KK^{\top},\qquad P_k=L_kL_k^{\top}$$

$$p(y=k\mid S)\propto\exp\left[\left\langle\lfloor K\rfloor-\lfloor L_k\rfloor,\lfloor A_k\rfloor\right\rangle+\frac{1}{4\theta}\left\langle K^{\frac{\theta}{2}}-L_k^{\frac{\theta}{2}},\,M^{-1}A_k\right\rangle\right]$$

$$\mathcal{L}_{P}[V]P+P\mathcal{L}_{P}[V]=V$$

$$\ell_k(S)=\left\langle\lfloor K\rfloor-\lfloor L_k\rfloor,\lfloor A_k\rfloor\right\rangle+\frac{1}{4\theta}\left\langle K^{\frac{\theta}{2}}-L_k^{\frac{\theta}{2}},M^{-1}A_k\right\rangle,\qquad p_k=\frac{e^{\ell_k}}{\sum_j e^{\ell_j}}$$

Mathematical statement

The paper parameterizes SPD matrices as S=KK^T and class prototypes as P_k=L_kL_k^T, where K and L_k are lower-triangular Cholesky factors and P_k is the prototype for class k. Its (theta,M)-BWCM classifier assigns probability proportional to the exponential of a score containing a Cholesky-coordinate term and a matrix-power term. The strictly lower-triangular operator floor(U) removes the diagonal and upper-triangular entries. A_k is a trainable lower-triangular tangent parameter, M is a positive-definite metric matrix, M^{-1}A_k is the metric-weighted tangent direction, and theta controls the matrix-power geometry. The Frobenius inner product is <U,V>=tr(U^T V). The Lyapunov operator L_P[V] is defined as the solution of L_P[V]P + P L_P[V] = V and is the reference differential operator that the Cholesky construction can avoid in its efficient implementation. The transferable asset is the product structure S=KK^T: positivity is guaranteed by construction and many operations can be performed on triangular factors rather than dense spectral coordinates.

Implementation notes

Integrate this as the final classifier or residual head after a network produces covariance features. Given a batch of symmetric features X, add diagonal jitter X_reg=X+delta I and compute K=chol(X_reg). Alternatively, have the upstream network emit unconstrained lower-triangular entries Z and define K=tril(Z), with K_ii=softplus(Z_ii)+delta; then S=KK^T is SPD by construction. Maintain one lower-triangular prototype factor L_k and tangent factor A_k per class. For every class, compute d1_k=sum_{i>j}(K_ij-L_k,ij)A_k,ij. Compute K^(theta/2) and L_k^(theta/2) with a stable SPD matrix-power routine; use a symmetric eigendecomposition for the baseline and the paper's Cholesky-based metric operators for the proposed implementation. Solve M q_k=A_k rather than explicitly forming M^{-1}, then evaluate the Frobenius inner product and the displayed logit formula. Apply ordinary cross-entropy to the logits. The Lyapunov equation L_P[V]P+P L_P[V]=V is the reference derivative; profile whether the proposed path avoids repeatedly solving it. Sweep delta, theta, and the condition number of M on validation data. First test on synthetic Gaussian covariance classification, followed by covariance-pooled CIFAR-10 or a human-activity covariance dataset. Compare Euclidean linear classification, log-Euclidean SPD classification, and this head. Measure accuracy, NaN frequency, wall-clock time, peak memory, and throughput as matrix dimension increases from 16 to 128. Success is comparable or higher accuracy with lower eigendecomposition overhead and stable gradients for nearly singular covariances.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a Cholesky-structured SPD classifier with automatic positive-definiteness, triangular tangent terms, and matrix-power logits in experiment.py. The mechanism checks matched the predictions: Cholesky reconstruction error was 8.9e-16, all factor-generated matrices had positive minimum eigenvalues, and both the power expansion error and scaled-power limit error decreased approximately linearly when theta was halved. On the synthetic task, the structured head and log-Euclidean baseline both reached 100% accuracy; the structured head had lower final loss (0.0638 vs 0.2670) but was substantially slower (1.58s vs 0.044s), so no speedup was observed.

Agent confidence: 8/10

Baseline: Log-Euclidean prototype classifier: accuracy 1.000, cross-entropy 0.2670, runtime 0.0436 s
Idea: Cholesky-structured head: accuracy 1.000, cross-entropy 0.0638, runtime 1.5751 s; reconstruction error 8.9e-16; minimum generated-factor eigenvalues 8.5e-05 to 3.9e-04; scaled-power limit error decreased from 0.01705 at theta=0.4 to 0.000979 at theta=0.025

Limitations: The experiment used small synthetic covariance matrices, M=I, full-batch optimization, and eigendecomposition for matrix powers, so it does not validate the proposed speed advantage of a genuinely Cholesky-only power/operator implementation. It did not test CIFAR-10, activity data, dimensions up to 128, minibatch throughput, peak memory, or repeated Lyapunov solves.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.19305) Sep 2, 2026

No significant difference vs. the tuned baseline (+4366.7%, p=0.2308, wins 3 / 8 paired seeds).

Benchmark: Spd_covariance_classification
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.0056
Idea mean: 0.2512
Effect (Δ): 0.2456 (+4366.7%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.2308 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±5101.1%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The custom track is synthetic rather than a real-world covariance dataset; matrix dimension was fixed at 5, M was fixed to the identity, and the matrix-power implementation used symmetric eigendecomposition rather than a genuinely Cholesky-only operator. Wall-clock speed, peak memory, and dimensions up to 128 were not benchmarked. The custom track registration into the shared read-only bench was not performed by this experiment.

How to run: python3 spd_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented a custom SPD covariance-classification track because no built-in track contains the required SPD covariance structure. The trained Cholesky-structured system preserved positive-definite prototype factors and finite logits, confirming the stability mechanism, but its mean test error was 0.25125 versus 0.005625 for the baseline; paired delta was +0.245625 with permutation p=0.2308, so there was no win.

Baseline: Mean test classification error 0.005625 across 8 seeds; best baseline sweep setting lr=0.001, epochs=18.
Idea: Mean test classification error 0.251250 across 8 seeds; selected idea setting lr=0.003, epochs=18; observed prototype minimum eigenvalue 0.420817 and finite-logit fraction 1.0.

Benchmark result