Geometrically Attracting Random Recurrent Layer

Integrate the mechanism into an RNN or state-space block with K candidate maps, such as Fₜ,ᵢ(hₜ,xₜ) = tanh(Wₜ,ᵢhₜ + Uₜ,ᵢxₜ + bₜ,ᵢ), and a time-dependent categorical gate pₜ,ᵢ. At each training step, estimate each candidate gain L̂ₜ,ᵢ using one to three power-iteration steps on the hidden-to-hidden Jacobian. A cheaper upper bound is ||Wₜ,ᵢ||₂ because tanh is 1-Lipschitz. Pseudocode: compute pₜ; sample a candidate or use the probability-weighted mixture; update hₜ₊₁ = Fₜ,ᴵₜ(hₜ,xₜ); estimate all L̂ₜ,ᵢ; compute gₜ = log(Σᵢ pₜ,ᵢL̂ₜ,ᵢ + epsilon); add λ times the squared positive part of gₜ − log(ρ*) to the task loss. Track γ̂ₜ = T⁻¹Σₜgₜ, and optionally rescale recurrent matrices when γ̂ₜ exceeds zero. The paper supplies the contraction mechanism; Jacobian estimates, routing probabilities, and finite-horizon exponents are measured empirically. First test on sequential MNIST and the adding problem against a GRU and an unconstrained random-map RNN with matched parameter counts. Run two identical input sequences from hidden states separated by unit norm, and record dₜ = ||hₜ − h′ₜ|| for 200 steps. The prediction is a stability boundary near γ̂ = 0: below it, log dₜ has negative slope approximately γ̂; above it, distances do not decay geometrically. The measured slope should agree with the accumulated log-gain within about 20 percent.