Unverified
2026
Bounded Adaptive Hebbian Fast-Weight Cache
Add a recurrent associative matrix to each selected transformer layer so recent key-value relationships can be retrieved without retaining every past token or performing gradient updates. The matrix uses input-dependent retention and write gates, but retrieval is always performed from the pre-write state, preventing the current target from leaking into its own prediction. Frobenius-norm clipping makes the recurrent memory bounded and provides a direct stability control.
Paper: Where Should Experience Live? Hierarchical Hebbian Memory for Continual Vision Transformers
arXiv:2609.00358