The Binomial Channel: On Capacity, Optimal Inputs, and Beta-Binomial Approximation

What the math gives to ML

The paper gives a principled stochastic bottleneck whose continuous input is converted into an n-trial count, together with an approximately capacity-optimal arcsine/Beta(1/2,1/2) input and an explicit beta-binomial output law. This can be transferred to neural networks as a population-count representation: a scalar probability produced by a network is encoded into a discrete count, preserving information while making the representation compact and uncertainty-aware. The paper's exact binomial identities also permit low-variance or exact gradients through the discrete channel when n is small. The strongest initial application is a variational bottleneck or classifier head where output entropy is encouraged toward the beta-binomial profile rather than allowing arbitrary, saturated count distributions.