# Cholesky-Structured SPD Classifier

- ID: 1145
- Canonical URL: https://synthcore.org/idea/1145/cholesky-structured-spd-classifier
- API JSON: https://synthcore.org/api/idea/1145.json
- API Markdown: https://synthcore.org/api/idea/1145.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.19305](https://arxiv.org/abs/2607.19305)
- Category: architecture
- Solves: speedup, stability, accuracy
- ML areas: graph-nn, cnn, mlp, training-dynamics, inference-speedup
- Math tags: geometry, linear-algebra, matrix-analysis, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Build an SPD classifier and residual head directly from Cholesky factors, using lower-triangular differences and matrix-power terms instead of generic eigendecomposition-based logarithm operators. This retains covariance geometry while making positive-definiteness automatic and backpropagation more numerically stable for minibatch training.

## Mathematical statement

The paper parameterizes SPD matrices as S=KK^T and class prototypes as P_k=L_kL_k^T, where K and L_k are lower-triangular Cholesky factors and P_k is the prototype for class k. Its (theta,M)-BWCM classifier assigns probability proportional to the exponential of a score containing a Cholesky-coordinate term and a matrix-power term. The strictly lower-triangular operator floor(U) removes the diagonal and upper-triangular entries. A_k is a trainable lower-triangular tangent parameter, M is a positive-definite metric matrix, M^{-1}A_k is the metric-weighted tangent direction, and theta controls the matrix-power geometry. The Frobenius inner product is <U,V>=tr(U^T V). The Lyapunov operator L_P[V] is defined as the solution of L_P[V]P + P L_P[V] = V and is the reference differential operator that the Cholesky construction can avoid in its efficient implementation. The transferable asset is the product structure S=KK^T: positivity is guaranteed by construction and many operations can be performed on triangular factors rather than dense spectral coordinates.

## Key formulas

- $$S=KK^{\top},\qquad P_k=L_kL_k^{\top}$$
- $$p(y=k\mid S)\propto\exp\left[\left\langle\lfloor K\rfloor-\lfloor L_k\rfloor,\lfloor A_k\rfloor\right\rangle+\frac{1}{4\theta}\left\langle K^{\frac{\theta}{2}}-L_k^{\frac{\theta}{2}},\,M^{-1}A_k\right\rangle\right]$$
- $$\mathcal{L}_{P}[V]P+P\mathcal{L}_{P}[V]=V$$
- $$\ell_k(S)=\left\langle\lfloor K\rfloor-\lfloor L_k\rfloor,\lfloor A_k\rfloor\right\rangle+\frac{1}{4\theta}\left\langle K^{\frac{\theta}{2}}-L_k^{\frac{\theta}{2}},M^{-1}A_k\right\rangle,\qquad p_k=\frac{e^{\ell_k}}{\sum_j e^{\ell_j}}$$

## Implementation notes

Integrate this as the final classifier or residual head after a network produces covariance features. Given a batch of symmetric features X, add diagonal jitter X_reg=X+delta I and compute K=chol(X_reg). Alternatively, have the upstream network emit unconstrained lower-triangular entries Z and define K=tril(Z), with K_ii=softplus(Z_ii)+delta; then S=KK^T is SPD by construction. Maintain one lower-triangular prototype factor L_k and tangent factor A_k per class. For every class, compute d1_k=sum_{i>j}(K_ij-L_k,ij)A_k,ij. Compute K^(theta/2) and L_k^(theta/2) with a stable SPD matrix-power routine; use a symmetric eigendecomposition for the baseline and the paper's Cholesky-based metric operators for the proposed implementation. Solve M q_k=A_k rather than explicitly forming M^{-1}, then evaluate the Frobenius inner product and the displayed logit formula. Apply ordinary cross-entropy to the logits. The Lyapunov equation L_P[V]P+P L_P[V]=V is the reference derivative; profile whether the proposed path avoids repeatedly solving it. Sweep delta, theta, and the condition number of M on validation data. First test on synthetic Gaussian covariance classification, followed by covariance-pooled CIFAR-10 or a human-activity covariance dataset. Compare Euclidean linear classification, log-Euclidean SPD classification, and this head. Measure accuracy, NaN frequency, wall-clock time, peak memory, and throughput as matrix dimension increases from 16 to 128. Success is comparable or higher accuracy with lower eigendecomposition overhead and stable gradients for nearly singular covariances.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a Cholesky-structured SPD classifier with automatic positive-definiteness, triangular tangent terms, and matrix-power logits in experiment.py. The mechanism checks matched the predictions: Cholesky reconstruction error was 8.9e-16, all factor-generated matrices had positive minimum eigenvalues, and both the power expansion error and scaled-power limit error decreased approximately linearly when theta was halved. On the synthetic task, the structured head and log-Euclidean baseline both reached 100% accuracy; the structured head had lower final loss (0.0638 vs 0.2670) but was substantially slower (1.58s vs 0.044s), so no speedup was observed.

### Mechanism check

- Verdict: Built a Cholesky-structured SPD classifier with automatic positive-definiteness, triangular tangent terms, and matrix-power logits in experiment.py. The mechanism checks matched the predictions: Cholesky reconstruction error was 8.9e-16, all factor-generated matrices had positive minimum eigenvalues, and both the power expansion error and scaled-power limit error decreased approximately linearly when theta was halved. On the synthetic task, the structured head and log-Euclidean baseline both reached 100% accuracy; the structured head had lower final loss (0.0638 vs 0.2670) but was substantially slower (1.58s vs 0.044s), so no speedup was observed.
- Confidence: 8/10
- Limitations: The experiment used small synthetic covariance matrices, M=I, full-batch optimization, and eigendecomposition for matrix powers, so it does not validate the proposed speed advantage of a genuinely Cholesky-only power/operator implementation. It did not test CIFAR-10, activity data, dimensions up to 128, minibatch throughput, peak memory, or repeated Lyapunov solves.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.2308
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/457/bench_report.json)
- [experiment.py](https://synthcore.org/code/457/experiment.py)
- [report.md](https://synthcore.org/code/457/report.md)
- [report_bench_2026-08-31T123600.md](https://synthcore.org/code/457/report_bench_2026-08-31T123600.md)
- [spd_bench.py](https://synthcore.org/code/457/spd_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/457)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.