Conditioned Direct Feedback Alignment via Activity and Error Geometry

What the math gives to ML

The paper identifies a concrete geometric failure mode of direct feedback alignment: each local update is an outer product whose anisotropy can arise independently from presynaptic activities or DFA local errors. Its transferable asset is a factorized, layer-local preconditioner that whitens one or both sides of this outer product using empirical second moments, without requiring a transposed-weight backward pass. The most promising implementation is two-sided damped nDFA, with separate activity and error damping because noisy error covariances can destabilize training. This is especially relevant to biologically local, decentralized, or communication-constrained training, and provides a falsifiable optimizer baseline for ordinary MLPs.