Stochastic trace estimation for positive trace-class operators

What the math gives to ML

The paper develops matrix-free trace estimators for positive trace-class operators, extending Hutchinson and Hutch++ to infinite-dimensional settings. The transferable asset is variance reduction from projecting out a low-rank subspace discovered through operator-vector products, reducing the number of expensive products needed for reliable trace estimation from roughly O(epsilon^-2) to O(epsilon^-1) in the positive-semidefinite case. A neural-network use is to estimate the trace of a damped Hessian or empirical Fisher through Hessian-vector products, enabling curvature-controlled learning-rate schedules or regularization without materializing the matrix. The most defensible first transfer is a Hutch++ curvature monitor whose predictions can be tested against ordinary Hutchinson at equal Hessian-vector-product cost.