Machine-learning ideas tagged Rnn in the ML taxonomy of the Math2NN corpus.
Add a functional-calculus regularizer to the transition operator of an RNN, linear state-space model, or deep-equilibrium layer. The regularizer uses polynomial probes to detect non-normal transient amplification that ordinary eigenvalue-radius penalties can miss.
Turn a recurrent or state-space memory into a constrained hereditary state: the latent state remains in a learned convex domain, and only input motion that reaches the boundary changes the plastic component. This creates a nonexpansive, rate-independent memory that should suppress unstable state growth and make the representation depend on meaningful cumulative changes rather than arbitrary update frequency.
Replace the unconstrained parameter update of a selected neural layer by a tangent update generated by a rank-two skew-symmetric operator. A Cayley transform then applies this operator while exactly preserving a quadratic parameter energy, preventing exploding or vanishing layer norms without projecting after every step. Add a separately trained scalar gain if fixed norm would otherwise reduce expressivity.
Replace a standard permutation-invariant object pool with a latent state on an unordered configuration together with a fiber vector transported along the observed object trajectories. The instantaneous state remains invariant to reordering, but loops and exchanges of objects act through learned monodromy matrices, allowing the network to represent path-dependent interactions without assigning arbitrary permanent object indices.
Replace a conventional leaky recurrent update with a population of stochastic membrane potentials that evolve only while subthreshold, emit an event at threshold, undergo a delayed reset, and receive feedback from a filtered population firing rate. Add a shared noise source alongside independent neuron noise to regularize the layer while preserving coordinated population-level dynamics.
Use the M phase-aligned parameterizations produced by cyclic reformulation as an empirical ensemble of neural dynamics rather than selecting one phase or averaging only predictions. Their centroid supplies a nominal model, while their convex hull defines a low-dimensional uncertainty set used for robust rollout training and uncertainty-aware inference.
Add a slow latent two-state gate to a recurrent, state-space, or world-model network so that separate experts represent two qualitatively different dynamical regimes. Train the gate using the paper's two-state population and fluctuation mechanism rather than allowing an unconstrained softmax to average incompatible regimes. The model should allocate extra capacity near the gate's susceptibility peak, where regime uncertainty and forecast variance are predicted to be largest.
Build a recurrent block as a fixed or learned ordering of local vertex foldings, mirroring the paper's identification of staircase solution maps with Coxeter elements of a folding group. Each folding changes one polygon coordinate by a rational cross-ratio completion while leaving all other coordinates unchanged. The resulting structured recurrence is reversible and can support constant-memory backpropagation by recomputing folds in reverse order.
Represent a hidden state as complex-valued points on a two-dimensional lattice and replace unconstrained local updates by the exact harmonic-quadrilateral completion rule from discrete conformal geometry. Given three corners of a plaquette, compute the fourth corner by a Mobius-rational formula enforcing cross-ratio minus one, then use a learned readout or forcing term for task-specific predictions. The layer supplies a hard geometric inductive bias and a directly measurable local constraint…
Use the paper's scattering energy balance as a measurable regularizer for an existing recurrent or state-space model instead of replacing its architecture. Penalize positive violations of the per-step energy inequality and, for paired examples, penalize violations of incremental passivity so that the model learns not to amplify perturbations over long sequences.
Build a recurrent or continuous-depth block from a dissipative vector field and project every state derivative onto the tangent cone of a closed convex hidden-state set. Unlike ordinary clipping, tangent-cone projection removes only the outward component at the boundary and preserves admissible motion. Under the paper's maximal-dissipativity result, the continuous flow is nonexpansive in its initial state.
In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…
Augment a neural router with the age of its current expert or latent regime and use an age-dependent hazard to determine when switching is likely. Unlike ordinary token-wise softmax routing, the router can learn non-geometric residence times, suppressing unstable expert oscillations while still allowing rapid transitions when the current regime becomes inappropriate.
Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.
Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.
Add a training-time regularizer that keeps the empirical joint covariance of hidden activations on multiple inputs close to the recursively predicted NNGP covariance. The regularizer targets the finite-width fluctuations quantified by the Wasserstein result, and is particularly appropriate for recurrent networks and attention blocks with shared weights, where hidden states at different positions or time steps are statistically coupled.
Represent each latent state as a Heisenberg-group element and replace Euclidean interpolation in an upsampling or recurrent transition block by a four-point horizontal refinement plus the exact central signed-area correction. The module preserves the geometry of noncommutative composition, allowing the central latent coordinate to encode path-dependent information that ordinary coordinate-wise interpolation discards.
Regularize a learned state-space transfer function so its matrix response has positive real part on sampled points in the unit disk and its associated reproducing-kernel Gram matrix is positive semidefinite. This provides a frequency-domain stability signal that complements rollout-based penalties and spectral-radius clipping.
Replace a generic recurrent transition with two coupled unitary transitions that share one block column and differ by a sign on the other block column. Each transition preserves hidden-state norm exactly, while the structured difference gives a controlled two-path recurrent architecture for long-context modeling.
Model the scalar feedback route in a recurrent layer as a rank-one perturbation of its open-loop transition. Regularize the frequency response of that route so that no mode reaches unit loop gain, directly targeting oscillatory and slowly decaying instabilities rather than relying only on gradient clipping.
Use the paper's third-order phase-locked-loop equations as a recurrent neuron instead of a leaky integrate-and-fire unit. Emit a spike whenever the phase crosses a chosen threshold, allowing one state trajectory to represent both slow burst envelopes and fast within-burst oscillations.
Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension \(d=\xi(G)\), whereas a discrete state encoding is lower-bounded by \(N\geq\chi(G)\). This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed…
Attach predictive distributions to successive information-update steps of a recurrent, state-space, iterative, or diffusion model and penalize violations of the measure-valued martingale condition. The model may become more certain as information arrives, but its later forecasts must not exhibit systematic conditional bias relative to earlier forecasts.
Add a two-sided cone-restricted spectral penalty to a recurrent or state-space model. Instead of estimating growth using a symmetric singular-value surrogate, jointly optimize a positive right vector and positive left vector in the extended quotient from the paper, targeting a real generalized eigenvalue of the learned non-selfadjoint transition operator.