Noise-Aware Soft ECOC Decoding

Use the same $N$-bit projection head, but retain real-valued logits $s_n$ rather than immediately taking signs. On a held-out calibration set, estimate each bit's conditional error rate $q_n$ by comparing the predicted sign with the target code bit: $\hat q_n=(\sum_i\mathbf{1}[\operatorname{sign}(s_{i,n})\neq c_{y_i,n}]+\alpha)/(M+2\alpha)$, with small Laplace smoothing $\alpha$. At inference, decode with weighted Hamming cost $\sum_n w_n\mathbf{1}[b_n\neq c_{k,n}]$, where $w_n=\log((1-\hat q_n)/\hat q_n)$, or use the soft log-likelihood expression in the formula and replace hard bits with Bernoulli probabilities $p_n=\sigma(s_n)$. During training, use binary cross-entropy but optionally apply dropout to bits according to a simulated crossover probability $q_n$, making the embedding robust to distributed packet loss or noisy quantization. The paper-derived quantities are the Gaussian correlation relation $q=\arccos(\rho)/\pi$ and the exponential pairwise-error factor; actual $q_n$ must be estimated empirically because neural bit errors are not identically distributed. First experiment on CIFAR-100 or ImageNet-100 with a ResNet-18, injecting independent bit flips at inference for rates $q\in\{0,0.01,0.05,0.1,0.2\}$. Compare unweighted Hamming, weighted Hamming, and soft likelihood decoding at the same $N$. Success is a flatter accuracy-versus-noise curve and recovery of baseline accuracy with fewer bits than unweighted decoding; also report calibration cost and decoder latency.