arXiv:2607.19334 2026 Architecture 2 ideas extracted · analyzed Aug 30, 2026
The paper gives a constructive random-coding principle for multiclass prediction using only binary decisions: random hyperplanes assign class centers binary codewords, and about $2\log_2 K+\log_2(1/\delta)$ tests suffice for $K$ classes with failure probability at most $\delta$ in high dimension. The transferable asset is the logarithmic dependence on the number of classes without optimizing a data-dependent codebook, together with a natural noisy-channel interpretation of binary classifier outputs. A practical neural implementation is a fixed or slowly refreshed random-hyperplane ECOC head on top of a learned embedding, decoded by Hamming or likelihood distance. This is most promising for large-class classification, distributed binary inference, and architectures where replacing a $K$-way softmax with $O(\log K)$ binary outputs reduces communication or classifier parameters.
Treat the binary outputs of the hyperplane head as a noisy channel and decode with reliability-weighted likelihood rather than unweighted Hamming distance. Estimate each bit's flip probability on validation data and give unreliable hyperplanes less influence, while retaining the logarithmic code-length scaling.
Replace a $K$-class softmax with $N$ binary hyperplane heads, where each class is represented by the signs of its projections onto fixed random directions. Train the embedding to reproduce these codewords and decode by nearest Hamming codeword. The paper's guarantee suggests that $N\approx 2\log_2 K+\log_2(1/\delta)$ can separate all class centers with high probability in sufficiently high dimension, giving a concrete width rule rather than choosing the number of binary heads heuristically.