Failed on benchmark
2026
Implicit proximal Hopfield retrieval
Replace an explicit attention step by an implicit Euler step that solves a proximal subproblem involving the Hopfield energy. The new state is evaluated inside the softmax self-consistently, which makes the method less sensitive to large step sizes and can prevent explicit attention from overshooting or tunneling between attraction basins.
Paper: Basin-Preserving Discretizations of Modern Hopfield Retrieval Dynamics: Energy Cells, Dissipation, and the Attention Limit
arXiv:2608.21304