Constrained Maximum Entropy Contiguous Aggregations

What the math gives to ML

This paper provides a discrete optimization problem over ordered partitions of a probability vector: aggregate consecutive masses into a fixed number of bins while making the resulting distribution's entropy as large as possible without exceeding a budget. The transferable asset is the monotonicity of entropy under contiguous merging, together with an explicit approximation-gap bound based on the order statistics of the original probabilities. A direct neural-network use is entropy-budgeted contiguous pooling of sequence tokens or key/value states, where attention mass determines the partition and the pooled representation preserves sequence order. This can turn a heuristic token-merging rule into a controllable compression mechanism with an interpretable entropy constraint.