Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks

What the math gives to ML

The paper's transferable contribution is a perturbation-learning rule that confines random exploration to the input-dependent subspace of an online objective rather than perturbing every reservoir state or parameter. Its key asset is dimensionality control: scalar-feedback estimators whose variance grows with input dimension can replace estimators whose variance grows with reservoir width. A practical neural-network adaptation is to construct an input-conditioned tangent map from a low-dimensional signal into the trainable parameters, estimate a two-sided loss directional derivative in that subspace, and update only through the same map. This is especially attractive for continual learning, neuromorphic hardware, and settings where backpropagation or stored activations are unavailable.