✗ Failed on benchmark 2026

Conditioned PSD sensing bottleneck

Implementation & benchmark of arXiv:2608.22418 — Optimal Condition Numbers in Low-Rank Positive Semidefinite Matrix Sensing

Usefulness6/10

Difficulty5/10

Novelty6/10

Regularization Stability Accuracy Memory Embedding Architecture Regularization Linear algebra Metric geometry Optimization

Source paper: Optimal Condition Numbers in Low-Rank Positive Semidefinite Matrix Sensing arXiv:2608.22418 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Represent an intermediate feature as a low-rank PSD matrix and compress it using nonnegative measurements $\langle A_i,X\rangle$, while penalizing the empirical ratio between maximum and minimum measurement distortion over low-rank feature pairs. This directly discourages collapsed directions and excessively amplified directions in a covariance or Gram-feature bottleneck.

Formulas

$$\Phi_{\mathcal{A}}(X):=(\langle A_{1},X\rangle,\dots,\langle A_{m},X\rangle),\qquad \langle A,X\rangle:=\operatorname{Tr}(A^{*}X).$$

$$L_{\mathcal A}(r):=\inf_{\substack{X,Y\succeq0,\;\operatorname{rank}(X),\operatorname{rank}(Y)\le r\\X\ne Y}}\frac{\|\Phi_{\mathcal A}(X)-\Phi_{\mathcal A}(Y)\|_1}{\|X-Y\|_F},\qquad U_{\mathcal A}(r):=\sup_{\substack{X,Y\succeq0,\;\operatorname{rank}(X),\operatorname{rank}(Y)\le r\\X\ne Y}}\frac{\|\Phi_{\mathcal A}(X)-\Phi_{\mathcal A}(Y)\|_1}{\|X-Y\|_F}.$$

$$\beta_{\mathcal A}(r):=\frac{U_{\mathcal A}(r)}{L_{\mathcal A}(r)}.$$

$$\mathcal L_{\mathrm{cond}}=\log(\widehat U+\varepsilon)-\log(\widehat L+\varepsilon),\quad q_j=\frac{\|\Phi_{\mathcal A}(X_j)-\Phi_{\mathcal A}(Y_j)\|_1}{\|X_j-Y_j\|_F+\varepsilon},\quad \widehat U=\operatorname{quantile}_{0.95}(\{q_j\}),\ \widehat L=\operatorname{quantile}_{0.05}(\{q_j\}).$$

Mathematical statement

The paper defines the PSD measurement map $\Phi_{\mathcal A}(X)=(\langle A_1,X\rangle,\ldots,\langle A_m,X\rangle)$, where $A_i\succeq0$, $X\succeq0$, and $\langle A,X\rangle=\operatorname{Tr}(A^*X)$. For rank at most $r$, the lower and upper $\ell_1$-to-Frobenius Lipschitz constants are $L_{\mathcal A}(r)=\inf_{X\ne Y}\|\Phi_{\mathcal A}(X)-\Phi_{\mathcal A}(Y)\|_1/\|X-Y\|_F$ and $U_{\mathcal A}(r)=\sup_{X\ne Y}\|\Phi_{\mathcal A}(X)-\Phi_{\mathcal A}(Y)\|_1/\|X-Y\|_F$, with both $X,Y$ PSD and rank at most $r$. The condition number is $\beta_{\mathcal A}(r)=U_{\mathcal A}(r)/L_{\mathcal A}(r)$. The neural adaptation uses minibatch estimates of these quantities; a large empirical $L$ prevents indistinguishable representations, while a small $U$ prevents unstable amplification.

Implementation notes

Use this as a replacement for a generic projection bottleneck in a model that naturally produces covariance or Gram features. At an intermediate layer, let $H\in\mathbb R^{d\times s}$ be a token-feature matrix, form $X=HH^\top/(s+\varepsilon)\in\mathbb S_d^+$, and parameterize each sensing matrix as $A_i=a_i a_i^\top$, so the bottleneck output is $z_i=\langle A_i,X\rangle=\|a_i^\top H\|_2^2/(s+\varepsilon)$. During training, retain two augmented or independently sampled views of each example, producing $X_j,Y_j$, and compute $q_j$ using the formula above. Use detached 5th and 95th percentile estimates for $\widehat L$ and $\widehat U$, add $\lambda\mathcal L_{\mathrm{cond}}$ to the task loss, and stop gradients through quantile selection if optimization is noisy. Pseudocode: H1,H2=model_features(view1,view2); X=H1@H1.T/(s+eps); Y=H2@H2.T/(s+eps); zX=(A@H1).square().sum(1)/s; zY=(A@H2).square().sum(1)/s; q=abs(zX-zY).sum()/(norm(X-Y,'fro')+eps); Lhat=quantile(q,.05); Uhat=quantile(q,.95); loss=task_loss+lambda*(log(Uhat+eps)-log(Lhat+eps)). First test on CIFAR-10 with a small ResNet or ViT, using $d=64,m=128$, against a learned dense projection and an unregularized PSD bottleneck. Measure validation accuracy, perturbation consistency, empirical condition ratio, and accuracy after reducing $m$. Success means a larger lower distortion and improved robustness at equal bottleneck width.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a rank-one PSD sensing bottleneck toy verifier with quantile condition loss and optimized sensing vectors. All three mechanism predictions manifested: beta was invariant under common scaling while L and U scaled by 3.7^2, beta decreased from 3.281 to 1.237 as m increased from 8 to 256, and beta increased from 1.321 to 9.408 as anisotropy gamma increased from 1 to 16. At m=32, conditioning reduced test beta from 1.799 to 1.367 and increased L from 2.608 to 3.035, so the proposed effect is real in this controlled setting.

Agent confidence: 9/10

Baseline: Random normalized PSD sensing, m=32: L_q05=2.6077, U_q95=4.6922, beta=1.7994
Idea: Optimized quantile-conditioned PSD sensing, m=32: L_q05=3.0349, U_q95=4.1487, beta=1.3670; train log-condition loss 0.6917 -> 0.2812

Limitations: Only synthetic rank-one PSD feature pairs were tested; no CIFAR-10 model, dense projection baseline, task accuracy, perturbation robustness, FLOPs, memory, or multi-seed evaluation was run. The width and anisotropy sweeps use empirical quantiles rather than exact global Lipschitz constants.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.22418) Sep 2, 2026

Significantly WORSE than the tuned baseline (+296.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 0.2421
Idea mean: 0.9611
Effect (Δ): 0.719 (+296.9%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±73.4%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in sequence track was tested; vision/CIFAR transfer, larger sensing widths, longer training, lambda sweeps, robustness metrics, and exact global PSD Lipschitz constants were not evaluated. The mechanism signature used one trained seed because the required primary comparison already used the full 8 paired seeds.

How to run: python3 stage2_psd_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented a matched sequence-track benchmark using a shared transformer-style encoder, comparing a dense bottleneck against the PSD sensing bottleneck with the quantile condition loss. The idea had substantially worse test MSE (0.9611 vs 0.2421; paired delta +0.7190; permutation p=0.0081), so it is a significant loss under the required criterion. The trained-model signature confirmed narrower q spread/better beta (1.54 vs 12.46) but did not confirm increased lower distortion, and this mechanism improvement did not translate into task performance.

Baseline: Sequence transformer, dense bottleneck, best lr=0.006: mean test MSE 0.24213635176420212, std 0.041836180617489935; sweep means lr=0.001/0.003/0.006: 0.2674381099641323/0.2363879233598709/0.21260840073227882.
Idea: Sequence PSD sensing bottleneck, lambda=0.03, best lr=0.006: mean test MSE 0.9611421003937721, std 0.2093114434157431; paired delta +0.71900574862957, 0/8 wins, permutation p=0.0081. Seed-0 signature: beta 1.5383 vs baseline 12.4596, L_q05 0.4970 vs 1.2279.

Benchmark result