Beyond Client Averaging: A Client-Independent Second-Order Stationary-Bias Component in Stochastic SCAFFOLD

What the math gives to ML

The paper identifies an explicit stationary-mean bias in constant-step stochastic SCAFFOLD that survives even with infinitely many fully participating clients. The transferable asset is the coefficient-level expansion: nonlinear curvature converts local stochastic second moments into a predictable displacement proportional to f'''(x*) sigma^2 gamma^2, with a known dependence on the number of local steps H. A practical adaptation is a curvature-and-noise-based bias compensator for scalar or coordinatewise federated optimization, validated first in homogeneous federated benchmarks before attempting multidimensional extensions.