# Fractional Mahalanobis radial head

- ID: 464
- Canonical URL: https://synthcore.org/idea/464/fractional-mahalanobis-radial-head
- API JSON: https://synthcore.org/api/idea/464.json
- API Markdown: https://synthcore.org/api/idea/464.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.06089](https://arxiv.org/abs/2607.06089)
- Category: architecture
- Solves: accuracy, generalization, stability
- ML areas: mlp, embedding, loss
- Math tags: statistics, geometry, approximation-theory, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.

## Mathematical statement

For class c in {0,1}, let z=f_theta(x) be a d-dimensional neural embedding, let mu_c be the class center, and let Sigma_c be a positive-definite covariance matrix. The squared Mahalanobis radius is D_c^2(z)=(z-mu_c)^T Sigma_c^{-1}(z-mu_c). For shared elliptical generator g, the paper's exact log-likelihood ratio is Lambda(x)=varphi(D_1^2(x))-varphi(D_0^2(x))+0.5 log(|Sigma_0|/|Sigma_1|)+log(pi_1/pi_0), where varphi=log g and pi_c are class priors. Adapt varphi with a finite fractional-power approximation h(t)=b_0+sum_{k=1}^K b_k t^{alpha_k}, using fixed exponents alpha_k>0 such as {0.25,0.5,1,1.5,2}. The resulting neural logit is l(x)=h(D_1^2(f_theta(x)))-h(D_0^2(f_theta(x)))+0.5 log(|Sigma_0|/|Sigma_1|)+log(pi_1/pi_0). The affine special case h(t)=b_0+b_1t is exactly the QDA-style baseline; noninteger powers provide curvature for heavy-tailed or non-Gaussian radial laws. Positive-definite covariances can be parameterized as Sigma_c=L_cL_c^T+epsilon I, with epsilon>0.

## Key formulas

- $$\Lambda(\bm{x})=\varphi(D_{1}^{2}(\bm{x}))-\varphi(D_{0}^{2}(\bm{x}))+\tfrac{1}{2}\log\frac{|\Sigma_{0}|}{|\Sigma_{1}|}+\log\frac{\pi_{1}}{\pi_{0}}.$$
- $$D_c^2(z)=(z-\mu_c)^T\Sigma_c^{-1}(z-\mu_c),\qquad \Sigma_c=L_cL_c^T+\varepsilon I.$$
- $$h(t)=b_0+\sum_{k=1}^{K}b_k t^{\alpha_k},\qquad \ell(x)=h(D_1^2(f_\theta(x)))-h(D_0^2(f_\theta(x)))+\frac12\log\frac{|\Sigma_0|}{|\Sigma_1|}+\log\frac{\pi_1}{\pi_0}.$$
- $$\mathcal{L}=\operatorname{BCE}(\sigma(\ell(x)),y)+\lambda_\Sigma\sum_{c\in\{0,1\}}\|\Sigma_c\|_F^2+\lambda_b\|b\|_2^2.$$

## Implementation notes

Integrate this module at the binary classifier's final logit, after an existing encoder f_theta(x), rather than adding another MLP layer. Use an embedding width of 16-128. Maintain two learnable centers mu_0 and mu_1 and lower-triangular factors L_0 and L_1; compute Sigma_c=L_c L_c^T+epsilon I with epsilon=1e-4 to 1e-3, and compute each radius by triangular solves instead of explicitly inverting Sigma_c. Choose exponents alpha=[0.25,0.5,1.0,1.5,2.0] and learn coefficients b. Initialize mu_c and covariance factors from one pass of encoder embeddings, initialize the affine coefficient near its QDA value, and initialize all non-affine coefficients near zero. For each minibatch, compute z=f_theta(x), then r_c=(z-mu_c)^T Sigma_c^{-1}(z-mu_c), evaluate h(r_c)=b_0+sum_k b_k*r_c**alpha_k, add determinant and prior terms, and train with binary cross-entropy. Freeze the encoder for 1-3 warmup epochs so centers and covariances stabilize, then unfreeze it with a learning rate 5-10 times smaller than the head. Use Cholesky solves, softplus diagonal entries of L_c, gradient clipping, and optionally clip radii at the 99.9th percentile for numerical safety. Compare against the same encoder with a linear head, a two-layer MLP head, and affine QDA on frozen embeddings. Start with UCI tabular data and synthetic Gaussian versus Student-t mixtures. Success means lower test log-loss and higher accuracy on heavy-tailed data at equal parameter count, no degradation on Gaussian data, improved calibration, and stable covariance condition numbers.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a differentiable fractional Mahalanobis radial head using Cholesky covariance solves, learned centers and covariances, fractional powers, determinant correction, and regularization. Cholesky radii matched explicit Mahalanobis computation to 4.4e-16, while the fractional basis approximated a Student-t radial log-generator with RMSE 0.039 over squared radii 1e-4–100. On the synthetic test, the idea slightly improved test log-loss over the MLP on Gaussian data (0.3014 vs 0.3025) and Student-t data (0.4285 vs 0.4326), but accuracy was essentially unchanged or slightly lower, so this is a modest promising signal rather than a decisive win.

### Mechanism check

- Verdict: Built a differentiable fractional Mahalanobis radial head using Cholesky covariance solves, learned centers and covariances, fractional powers, determinant correction, and regularization. Cholesky radii matched explicit Mahalanobis computation to 4.4e-16, while the fractional basis approximated a Student-t radial log-generator with RMSE 0.039 over squared radii 1e-4–100. On the synthetic test, the idea slightly improved test log-loss over the MLP on Gaussian data (0.3014 vs 0.3025) and Student-t data (0.4285 vs 0.4326), but accuracy was essentially unchanged or slightly lower, so this is a modest promising signal rather than a decisive win.
- Confidence: 6/10
- Limitations: Only a 2D synthetic identity embedding and one fixed random seed were tested. UCI data, a learned encoder, warmup scheduling, calibration, parameter/FLOP matching, repeated-seed confidence intervals, and explicit covariance condition-number tracking were not evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.01485
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1311/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1311/bench_report.json)
- [experiment.py](https://synthcore.org/code/1311/experiment.py)
- [report.md](https://synthcore.org/code/1311/report.md)
- [report_bench_2026-09-04T134853.md](https://synthcore.org/code/1311/report_bench_2026-09-04T134853.md)
- [results.json](https://synthcore.org/code/1311/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1311)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.