Solomonoff Induction and Singular Integrals

What the math gives to ML

The paper gives a constructive bridge between universal algorithmic priors and Bayesian model evidence: a monotone-machine semimeasure can reproduce the evidence of any computable model up to a constant factor. The transferable asset is the singular-learning asymptotic, in which the complexity penalty is not parameter count but the learning coefficient \(\lambda\) and multiplicity \(m\), invariants of the model's local geometry near optimal parameters. This suggests a neural architecture-selection and pruning criterion that measures how quickly each candidate's marginal likelihood grows with dataset size, favoring models with low singular complexity rather than merely few weights. The practical route is to estimate evidence for a small set of candidate networks using posterior samples, fit the \(\log n\) scaling, and use the estimated coefficient to select widths, ranks, or experts.