Fading-Memory Contractive Recurrent Block

1. Integration point: implement a recurrent cell or continuous-time state-space layer with hidden state h and one additional state m of equal dimension. Use a gated or spectrally normalized recurrent map F, then add kappa times (m minus h) before discretization. With step size Delta t, use h_new equal to h plus Delta t times [F(h,u) plus kappa(m-h)], and m_new equal to exp(-beta Delta t) times m plus [one minus exp(-beta Delta t)] times h_new. 2. Pseudocode: compute q equal to F_theta(h,u); compute h_new from the Euler equation; compute m_new from the exact exponential filter; return both states. Initialize m equal to h. 3. The paper-derived components are the history augmentation and the energy functional. Estimate the local Jacobian gain with one to three power iterations on the Jacobian of F with respect to h per batch; do not assume the weight spectral norm exactly equals the nonlinear Jacobian norm. Log perturbation growth, hidden-state norms, and the estimated largest real characteristic root. 4. First cheap experiment: compare a vanilla tanh RNN, a spectrally normalized RNN, and this fading-memory cell on the synthetic copy task and permuted-MNIST. Run two identical inputs from states separated by a perturbation of size 10^-5. On a scalar linearized cell, numerically calculate the roots of the stated polynomial and compare them with the observed slope of log perturbation norm versus time. The predicted signature is a transition from exponential amplification to decay as the recurrent gain crosses the root-zero boundary; the observed transition should be within 20 percent of the predicted gain threshold.