Delay-Resonance Monitor for Oscillatory Hidden States

1. Integration point: use a GRU, RNN, or linear state-space model with a FIFO buffer of hidden states. Define a delayed residual update \(h_{t+1}=h_t+\Delta t\,f_\theta(h_{t-d})\), or include both \(h_{t-d}\) and \(h_{t-2d}\) as inputs. Choose a reference hidden state \(\bar h\), usually the batch mean or zero after centering, and compute the Jacobian \(J_h=\partial f_\theta/\partial h\vert_{\bar h}\) with automatic differentiation. 2. Pseudocode: for each candidate frequency \(\omega_k=(2k+1)\pi/(2\tau)\), form \(C_k=i\omega_k I-e^{-i\omega_k\tau}J_h\); calculate \(r=\min_k\sigma_{\min}(C_k)\). For two delays replace \(C_k\) by \(i\omega_kI-(e^{-i\omega_k\tau}+e^{-i2\omega_k\tau})J_h\). Add \(\alpha\max(0,m-r)^2\) to the loss when resonance avoidance is desired, or add a narrow-band reward for the target mode when periodic memory is desired. 3. Computed from the paper: allowed frequencies, phase factors, and characteristic matrices. Estimated empirically: \(J_h\), singular values, and the actual hidden-state power spectrum. 4. First cheap experiment: train a 64-unit delayed RNN on adding-periodic-signals and copy-memory tasks, comparing vanilla training with resonance avoidance. Sweep recurrent gain or scalar \(\lambda\), record \(r\) and hidden-state Fourier power. The quantitative prediction is that oscillatory hidden activity and loss instability increase sharply when \(r\) approaches zero; the empirically observed transition in gain should occur within 20% of the gain predicted by \(\sigma_{\min}(C_k)=0\). For a targeted periodic task, the dominant spectral peak should occur at the selected \(\omega_k\) and its period should be within one discretization bin of \(2\pi/\omega_k\).