Performance evaluation of branch-free fused multiply-add algorithms for multi-component-type multiple-precision floating-point arithmetic

What the math gives to ML

The paper provides a concrete branch-free fused multiply-add for multi-component floating-point numbers, combining error-free multiplication and addition while avoiding data-dependent branches. This is transferable to mixed-precision neural-network training as a compensated accumulator for reductions, dot products, attention scores, and optimizer state, especially when FP16/BF16 inputs are accumulated on FP32 hardware. The key asset is the exact error decomposition supplied by TwoSum and TwoProd, which permits a small fixed number of extra words and predictable GPU execution. A practical first target is a two-word accumulator in GEMM or attention reductions, with the safer normalization path enabled when residuals overlap.