Solving Stochastic Fixed-Point Equations with High Probability

What the math gives to ML

The paper develops a stochastic fixed-point solver for nonexpansive or contractive operators when oracle noise has only bounded second moments, rather than strong tail assumptions. Its key transferable mechanism is recursive variance reduction using clipped differences between oracle evaluations at consecutive iterates, with the clipping radius proportional to the operator's Lipschitz scale and iterate displacement. This preserves local signal while controlling heavy-tailed increments. The most direct neural application is a stochastic deep-equilibrium or implicit layer whose repeated map evaluations use minibatches, dropout, Monte Carlo samples, or other noisy oracles.