# Equiangular tight-frame classifier head

- ID: 336
- Canonical URL: https://synthcore.org/idea/336/equiangular-tight-frame-classifier-head
- API JSON: https://synthcore.org/api/idea/336.json
- API Markdown: https://synthcore.org/api/idea/336.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.01564](https://arxiv.org/abs/2607.01564)
- Category: architecture
- Solves: accuracy, stability, generalization
- ML areas: embedding, loss, regularization, initialization
- Math tags: linear-algebra, geometry, information-theory, optimization
- Ratings: usefulness 7/10; difficulty 4/10; novelty 5/10

## Idea description

Replace unconstrained final classifier prototypes with an equiangular tight frame (ETF), or initialize them as an ETF and softly preserve the structure during training. The frame gives every class the same norm, an isotropic aggregate geometry, and equal pairwise interference, which should improve conditioning and reduce class-prototype collapse in normalized-softmax or contrastive models. For arbitrary class counts where an exact ETF is unavailable, optimize differentiable tight-frame and Welch-bound penalties.

## Mathematical statement

The paper defines maximal quantum leakage by Q(X to A)_rho = sup_{F_y} log(sum_y max_x tr(rho^x F_y)), where rho^x is the encoding of class or data x and {F_y} is a POVM. The transferable object is the optimal encoding geometry rather than the quantum measurement: represent each class by a unit vector w_i in R^d, with N classes and W=[w_1,...,w_N] in R^(d x N). A tight frame has isotropic frame operator WW^T=(N/d)I_d, where I_d is the d x d identity. An equiangular frame additionally has constant absolute pairwise coherence |w_i^T w_j|=c for every i != j, with the Welch lower bound c >= c_(N,d)=sqrt((N-d)/(d(N-1))) when N>d; equality is the ETF condition. The paper's explicit SIC construction for d=2,N=4 is psi_0=|0> and psi_k=(1/sqrt(3))|0>+sqrt(2/3) exp(i 2 pi(k-1)/3)|1> for k=1,2,3, having coherence 1/sqrt(3) and an isotropic average state. For real networks, use real simplex ETFs when N <= d+1, or numerically optimize the two ETF conditions for general N.

## Key formulas

- $$\mathcal{Q}(X\to A)_\rho:=\sup_{\{F_y\}_{y\in\mathcal{Y}}}\log\left(\sum_{y\in\mathcal{Y}}\max_{x\in\mathcal{X}}\operatorname{tr}(\rho^xF_y)\right).$$
- $$WW^\top=\frac{N}{d}I_d,\qquad \|w_i\|_2=1\quad(i=1,\ldots,N).$$
- $$|w_i^\top w_j|=c_{N,d}=\sqrt{\frac{N-d}{d(N-1)}}\quad\text{for all }i\ne j.$$
- $$\mathcal{L}_{\mathrm{ETF}}=\lambda_{\mathrm{tight}}\left\|\frac{d}{N}WW^\top-I_d\right\|_F^2+\lambda_{\mathrm{coh}}\frac{1}{N(N-1)}\sum_{i\ne j}\left((w_i^\top w_j)^2-c_{N,d}^2\right)^2.$$

## Implementation notes

Integrate this at the final classifier or metric-learning prototype matrix, not in the backbone. Let the backbone produce z in R^d; normalize it as z_hat=z/(||z||_2+epsilon), store one prototype w_i per class in W in R^(d x N), normalize each column, and compute logits ell_i=s z_hat^T w_i followed by ordinary cross-entropy. First construct a simplex ETF when N <= d+1: form A=I_N-(1/N)11^T, take an orthonormal basis Q in R^(N x d) for the centered subspace, and set W=sqrt(N/(N-1)) Q^T A followed by column normalization. For N>d+1, initialize W randomly and optimize the explicit ETF penalty jointly with the network; use c_(N,d)^2=(N-d)/(d(N-1)). Pseudocode: `z=normalize(backbone(x)); W=normalize_columns(W); logits=s*z@W; loss=CE(logits,y)+L_ETF(W)`. Compute the tight-frame term directly from WW^T and the coherence term from the Gram matrix G=W^T W; no quantum objects or POVM optimization are required. Keep W fixed for the first ablation, then compare learnable-with-penalty and unconstrained baselines. Start with CIFAR-100 and a small ResNet-18 or ViT-Tiny, using d=128 or 256, and compare standard cosine-softmax, ETF-fixed, and ETF-regularized heads at equal training FLOPs. Measure validation accuracy, minority-class accuracy, singular-value spread of WW^T, prototype coherence, gradient-norm variance, and calibration. Success would be higher accuracy or robustness at equal compute, a substantially flatter frame-operator spectrum, and fewer collapsed or highly correlated class prototypes; for fixed ETF heads, also test whether convergence is less sensitive to classifier initialization and learning rate.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Implemented an exact real simplex/orthonormal ETF constructor, diagnostics, mechanism sweeps, and a fixed-ETF versus learnable unconstrained cosine-softmax toy classifier. The three predicted geometric effects manifested: maximum tightness residual was 1.38e-15 versus 0, Welch-coherence error was 4.44e-16, and ETF frame-spectrum spread was below 2.30e-15 while random-head means ranged from 0.92 to 3.19. The fixed ETF did not improve this toy classifier (accuracy 0.6625 versus 0.6958 baseline; loss 0.6807 versus 0.6537), so the geometric mechanism is confirmed but an accuracy win is not.

### Mechanism check

- Verdict: Implemented an exact real simplex/orthonormal ETF constructor, diagnostics, mechanism sweeps, and a fixed-ETF versus learnable unconstrained cosine-softmax toy classifier. The three predicted geometric effects manifested: maximum tightness residual was 1.38e-15 versus 0, Welch-coherence error was 4.44e-16, and ETF frame-spectrum spread was below 2.30e-15 while random-head means ranged from 0.92 to 3.19. The fixed ETF did not improve this toy classifier (accuracy 0.6625 versus 0.6958 baseline; loss 0.6807 versus 0.6537), so the geometric mechanism is confirmed but an accuracy win is not.
- Confidence: 9/10
- Limitations: Only exact real ETF cases with d=N or d=N-1 were constructed; arbitrary N>d+1 differentiable ETF penalty optimization was not implemented. The classifier test used one tiny synthetic five-class dataset, one seed, a fixed ETF ablation only, training-set accuracy rather than a held-out benchmark, and no calibration, minority-class, FLOP, or gradient-variance evaluation.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.9528
- Paired wins: 3/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/114/bench_report.json)
- [custom_etf_track.py](https://synthcore.org/code/114/custom_etf_track.py)
- [etf_experiment.py](https://synthcore.org/code/114/etf_experiment.py)
- [report.md](https://synthcore.org/code/114/report.md)
- [report_bench_2026-08-30T145120.md](https://synthcore.org/code/114/report_bench_2026-08-30T145120.md)
- [results.json](https://synthcore.org/code/114/results.json)
- [run_bench.py](https://synthcore.org/code/114/run_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/114)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.