Fractional Mahalanobis radial head
Implementation & benchmark of arXiv:2607.06089 — Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis
Source paper: Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis arXiv:2607.06089 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.
Formulas
Mathematical statement
For class c in {0,1}, let z=f_theta(x) be a d-dimensional neural embedding, let mu_c be the class center, and let Sigma_c be a positive-definite covariance matrix. The squared Mahalanobis radius is D_c^2(z)=(z-mu_c)^T Sigma_c^{-1}(z-mu_c). For shared elliptical generator g, the paper's exact log-likelihood ratio is Lambda(x)=varphi(D_1^2(x))-varphi(D_0^2(x))+0.5 log(|Sigma_0|/|Sigma_1|)+log(pi_1/pi_0), where varphi=log g and pi_c are class priors. Adapt varphi with a finite fractional-power approximation h(t)=b_0+sum_{k=1}^K b_k t^{alpha_k}, using fixed exponents alpha_k>0 such as {0.25,0.5,1,1.5,2}. The resulting neural logit is l(x)=h(D_1^2(f_theta(x)))-h(D_0^2(f_theta(x)))+0.5 log(|Sigma_0|/|Sigma_1|)+log(pi_1/pi_0). The affine special case h(t)=b_0+b_1t is exactly the QDA-style baseline; noninteger powers provide curvature for heavy-tailed or non-Gaussian radial laws. Positive-definite covariances can be parameterized as Sigma_c=L_cL_c^T+epsilon I, with epsilon>0.
Implementation notes
Integrate this module at the binary classifier's final logit, after an existing encoder f_theta(x), rather than adding another MLP layer. Use an embedding width of 16-128. Maintain two learnable centers mu_0 and mu_1 and lower-triangular factors L_0 and L_1; compute Sigma_c=L_c L_c^T+epsilon I with epsilon=1e-4 to 1e-3, and compute each radius by triangular solves instead of explicitly inverting Sigma_c. Choose exponents alpha=[0.25,0.5,1.0,1.5,2.0] and learn coefficients b. Initialize mu_c and covariance factors from one pass of encoder embeddings, initialize the affine coefficient near its QDA value, and initialize all non-affine coefficients near zero. For each minibatch, compute z=f_theta(x), then r_c=(z-mu_c)^T Sigma_c^{-1}(z-mu_c), evaluate h(r_c)=b_0+sum_k b_k*r_c**alpha_k, add determinant and prior terms, and train with binary cross-entropy. Freeze the encoder for 1-3 warmup epochs so centers and covariances stabilize, then unfreeze it with a learning rate 5-10 times smaller than the head. Use Cholesky solves, softplus diagonal entries of L_c, gradient clipping, and optionally clip radii at the 99.9th percentile for numerical safety. Compare against the same encoder with a linear head, a two-layer MLP head, and affine QDA on frozen embeddings. Start with UCI tabular data and synthetic Gaussian versus Student-t mixtures. Success means lower test log-loss and higher accuracy on heavy-tailed data at equal parameter count, no degradation on Gaussian data, improved calibration, and stable covariance condition numbers.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 6/10
Built a differentiable fractional Mahalanobis radial head using Cholesky covariance solves, learned centers and covariances, fractional powers, determinant correction, and regularization. Cholesky radii matched explicit Mahalanobis computation to 4.4e-16, while the fractional basis approximated a Student-t radial log-generator with RMSE 0.039 over squared radii 1e-4–100. On the synthetic test, the idea slightly improved test log-loss over the MLP on Gaussian data (0.3014 vs 0.3025) and Student-t data (0.4285 vs 0.4326), but accuracy was essentially unchanged or slightly lower, so this is a modest promising signal rather than a decisive win.
- Agent confidence
- 6/10
- Baseline
- MLP test log-loss/accuracy: Gaussian 0.3025/87.13%; Student-t 0.4326/82.33%. Frozen QDA: Gaussian 0.3011/87.00%; Student-t 0.5266/81.20%.
- Idea
- Fractional radial test log-loss/accuracy: Gaussian 0.3014/86.93%; Student-t 0.4285/82.20%. Maximum test radius: Gaussian 24.74; Student-t 84.57. Cholesky-versus-inverse radius error: 4.4e-16.
Limitations: Only a 2D synthetic identity embedding and one fixed random seed were tested. UCI data, a learned encoder, warmup scheduling, calibration, parameter/FLOP matching, repeated-seed confidence intervals, and explicit covariance condition-number tracking were not evaluated.
How to run: python3 experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.06089)
Significantly WORSE than the tuned baseline (+3.5%, p=0.0149, wins 0 / 8 paired seeds).
- Benchmark
- Custom:heavy_tailed_binary_radial
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 0.1787
- Idea mean
- 0.185
- Effect (Δ)
- 0.0063 (+3.5%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0149 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±2.4%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the custom heavy-tailed binary classification track was tested because the four built-in tracks lack the idea's binary class-conditional radial likelihood structure. The custom dataset uses 400 training and 400 test samples, one tiny MLP encoder, 28 epochs, and three shared learning rates; no CIFAR/UCI benchmark, calibration analysis, parameter/FLOP audit, covariance condition-number report, or broader architecture sweep was run.
How to run: python3 bench_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
Built a local heavy-tailed binary-classification track and compared end-to-end matched systems using the same encoder, training budget, and shared learning-rate sweep. The fractional radial head showed strong trained-model nonlinear curvature beyond an affine radius difference, but its test error was significantly worse: mean 0.1850 versus baseline 0.17875, paired delta +0.00625, permutation p=0.01485. Therefore the mechanism is present, but it did not transfer into a benchmark win.
- Baseline
- 8-seed test error mean 0.17875, std 0.01447; baseline sweep best lr=0.003 (4-seed sweep mean 0.174375). Per-seed: [0.1675, 0.1800, 0.1825, 0.1675, 0.1725, 0.1925, 0.1600, 0.2075].
- Idea
- 8-seed test error mean 0.18500, std 0.01500; best idea lr=0.003 from shared grid. Per-seed: [0.1675, 0.1950, 0.1850, 0.1775, 0.1750, 0.2025, 0.1675, 0.2100]. Paired delta +0.00625; idea wins 0/8; p=0.01485; verdict idea worse (significant).
Benchmark result
Significantly WORSE than the tuned baseline (+3.5%, p=0.0149, wins 0 / 8 paired seeds).
- Benchmark
- Custom:heavy_tailed_binary_radial
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 0.1787
- Idea mean
- 0.185
- Effect (Δ)
- 0.0063 (+3.5%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0149 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±2.4%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the custom heavy-tailed binary classification track was tested because the four built-in tracks lack the idea's binary class-conditional radial likelihood structure. The custom dataset uses 400 training and 400 test samples, one tiny MLP encoder, 28 epochs, and three shared learning rates; no CIFAR/UCI benchmark, calibration analysis, parameter/FLOP audit, covariance condition-number report, or broader architecture sweep was run.
How to run: python3 bench_experiment.py
Artifacts
- bench_experiment.py 6.5 KB View Raw
- bench_report.json 2.7 KB View Raw
- experiment.py 7.8 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-09-04T134853.md 4.8 KB View
- results.json 1.3 KB View Raw