Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware

What the math gives to ML

The paper provides a simple, hardware-relevant sparsification operator rather than merely pruning weights: it removes low-magnitude activation events while retaining both positive and negative outliers. The transferable asset is the trainable, per-projection dead zone, which can convert dense matrix operations into event-driven sparse computation without collapsing the activation distribution to only positive values. A practical adaptation is to insert this operator after quantized linear projections in an SSM or transformer and train its thresholds with a smooth surrogate gradient, explicitly measuring the tradeoff between activation sparsity and task loss. The main uncertainty is whether GPU implementations benefit, but the method is directly testable for neuromorphic hardware, sparse kernels, and activation-aware inference.