✗ Failed on benchmark 2026

Calibrated Compact-Support Anomaly Score

Implementation & benchmark of arXiv:2607.16547 — Projective Maximum Entropy: Universality and Acceptance-Region Calibration

Usefulness7/10

Difficulty3/10

Novelty7/10

Regularization Accuracy Stability Generalization Loss Embedding Regularization Information theory Probability Statistics Geometry

Source paper: Projective Maximum Entropy: Universality and Acceptance-Region Calibration arXiv:2607.16547 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace Gaussian Mahalanobis scoring with the projective maximum-entropy density whose support is exactly a prescribed ellipsoid. The score is finite inside the admissible region and assigns an explicit boundary penalty outside it, avoiding arbitrary post-hoc Gaussian truncation.

Formulas

$$\gamma_R=\frac{2}{R^2-d-2},\qquad R^2>d+2.$$

$$r^2(x)=(x-\mu)^\top\Sigma^{-1}(x-\mu),\qquad p_R(x)=C_{d,R,\gamma}\left[1-\frac{r^2(x)}{R^2}\right]_+^{1/\gamma_R}.$$

$$C_{d,R,\gamma}=\frac{\Gamma\!\left(\frac d2+1+\frac1{\gamma_R}\right)}{\pi^{d/2}R^d\Gamma\!\left(1+\frac1{\gamma_R}\right)}.$$

$$-\log p_R(x)=\begin{cases}-\log C_{d,R,\gamma}-\frac1{\gamma_R}\log\left(1-r^2(x)/R^2\right),&r^2(x)<R^2,\\+\infty,&r^2(x)\ge R^2.\end{cases}$$

Mathematical statement

For dimension $d$ and squared Mahalanobis acceptance radius $R^2>d+2$, the paper calibrates the deformation as $\gamma_R=2/(R^2-d-2)$. Let $\mu\in\mathbb R^d$ be the reference location, $\Sigma\succ0$ the reference covariance, and $r^2(x)=(x-\mu)^\top\Sigma^{-1}(x-\mu)$. The compact maximum-entropy density is $p_R(x)=C_{d,R,\gamma}[1-r^2(x)/R^2]_+^{1/\gamma_R}$, where $[u]_+=\max(u,0)$, and its support is exactly $r^2(x)\le R^2$. The normalization is $C_{d,R,\gamma}=\Gamma(d/2+1+1/\gamma_R)/(\pi^{d/2}R^d\Gamma(1+1/\gamma_R))$. The calibration gives $\mathbb E[r^2]=d$, hence covariance $\Sigma$, while affine transformations of $(\mu,\Sigma)$ preserve the Mahalanobis geometry.

Implementation notes

Integrate this at the anomaly-scoring head of an embedding model, after a frozen or trainable encoder produces $z=f_\theta(x)\in\mathbb R^d$. Estimate robust $\mu$ and $\Sigma$ on nominal training embeddings, using a robust covariance estimator or shrinkage covariance $\Sigma_\lambda=(1-\lambda)\hat\Sigma+\lambda I$. Choose an acceptance radius from the desired nominal coverage, while enforcing $R^2>d+2$, and compute $\gamma_R=2/(R^2-d-2)$. For every embedding, solve $v=L^{-1}(z-\mu)$ with Cholesky factor $LL^\top=\Sigma$, then set $r^2=\|v\|^2$. Return the negative log-density for in-support points. During training, replace infinity by a stable barrier: clamp $1-r^2/R^2$ to $10^{-6}$, and add $\operatorname{softplus}(r^2-R^2)$ for outside points. Pseudocode: estimate $(\mu,\Sigma)$; compute $R,\gamma_R,C$; encode a batch; compute Mahalanobis radii; evaluate the piecewise score; optimize the task loss plus $\alpha$ times the score on nominal data. The paper supplies the radius/deformation and covariance-preserving construction; coverage quantiles, shrinkage, and soft barriers are empirical. First test on CIFAR-10 or MVTec with a small ResNet and compare Gaussian Mahalanobis and Gaussian-plus-truncation at equal features. Measure AUROC and false-positive rate at fixed nominal coverage. Success is improved OOD detection or calibration without changing the encoder.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built the calibrated compact-support anomaly scorer with Cholesky Mahalanobis geometry, exact normalization, calibrated gamma, hard outside-support scores, and a stable barrier option. Across R²={7.5,11,25,85} in d=5, numerical normalization was 1.0, sampled E[r²] was 5.003, 5.003, 4.998, 5.000 versus predicted 5, and boundary slopes matched predictions within 1.2e-6; affine radius error was 7.1e-15. The mechanism is real, but the equal-feature OOD comparison was exactly tied with Gaussian Mahalanobis (AUROC 0.7885 and anomaly acceptance 0.5445 at 95% nominal coverage), as expected from monotonicity in r².

Agent confidence: 9/10

Baseline: Gaussian Mahalanobis: AUROC=0.788464; anomaly acceptance at 95% nominal threshold=0.5445.
Idea: Compact calibrated score: AUROC=0.788464; anomaly acceptance at 95% nominal threshold=0.5445. Math checks: normalization≈1, E[r²]≈5 for all four radii, boundary slope absolute errors 7.6e-9 to 1.2e-6, affine radius error=7.1e-15.

Limitations: Only a synthetic equal-feature anomaly experiment was tested; no CIFAR-10, MVTec, learned encoder, robust covariance estimation, training-time barrier optimization, runtime/FLOP study, or real-world calibration evaluation was performed. The Monte Carlo covariance check used identity covariance and exact sampling rather than estimating parameters from finite nominal data.

How to run: python3 compact_anomaly.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.16547) Sep 2, 2026

Significantly WORSE than the tuned baseline (+45.0%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Geometry support regression
Model: matched_mlp_embedding_regressor
Paired seeds: 8
Baseline mean: 0.0503
Idea mean: 0.073
Effect (Δ): 0.0226 (+45.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±9.7%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Used the registered geometry-support regression track rather than real image OOD detection. CIFAR-10/MVTec, robust finite-sample covariance estimation, pretrained encoders, AUROC, and deployment-time anomaly detection were not tested.

How to run: python3 stage2_compact_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Ran the registered geometry_support_regression benchmark with matched MLP regressors, a tuned baseline sweep, and 8 paired seeds. The baseline achieved mean MSE 0.0503475652 at lr=0.006 versus 0.0729937740 for the idea; paired delta was +0.0226462088 with permutation p=0.0081, so the idea was significantly worse. The trained-model mechanism signature was not confirmed because observed mean radius-squared was 2.61810413 rather than the predicted 5.0.

Baseline: Best lr=0.006; sweep means were 0.0730441455 at lr=0.001, 0.0594160510 at lr=0.003, and 0.0495788129 at lr=0.006; full 8-seed mean MSE=0.0503475652.
Idea: Full 8-seed mean MSE=0.0729937740; paired delta=+0.0226462088; permutation p=0.0081; idea wins=0/8.

Benchmark result