Efficient Computation of Arbitrary-Order Directional Derivatives in Multiple Directions via Generalized Dual Numbers
arXiv:2608.15345
2026
Training
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper provides a truncated commutative algebra whose coefficient propagation computes arbitrary-order repeated directional derivatives without materializing derivative tensors. Its most transferable asset is the combination of factorial-scaled generalized dual arithmetic with polarization/inclusion–exclusion, which reconstructs mixed derivatives from evaluations along sums of directions. In neural networks this enables a small-order curvature or higher-order regularizer, diagnostic, or optimizer component using only a handful of forward-mode passes, especially when the parameter dimension is huge but the number of directions is small.
Ideas from this paper
Unverified
2026
Use generalized dual numbers to compute second- or third-order derivatives of the training loss along several parameter-space directions, then use polarization to recover mixed directional derivatives without forming a Hessian or third-order tensor. Add a bounded mixed-curvature penalty or use the resulting directional curvature to rescale updates in directions that are simultaneously sharp.
Useful6/10
Difficulty6/10
Novelty6/10