ML: Rnn

Machine-learning ideas tagged Rnn in the ML taxonomy of the Math2NN corpus.

583 ideas found

Unverified 2026

Finite-Plant Minimax RNN

Replace a single recurrent transition with a finite bank of candidate positive linear transitions and use a minimax controller to choose the feedback action at every time step. The controller evaluates candidate successors, selects the action whose worst-case predicted cost is smallest, and clips the action to preserve nonnegative hidden states. This should make an SSM or RNN less sensitive to transition-matrix mismatch and long-horizon disturbances.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Minimax adaptive control for finite sets of positive linear systems arXiv:2607.26816
Unverified 2026

Phase-retrieval observability regularizer

When the Schrödinger generator is learned, regularize its spectrum and eigenvectors so that the magnitude trajectory remains well-conditioned for recovering hidden complex states. Penalize small singular values of the squared-eigenvector matrix and near-colliding eigenvalue pair sums, preventing a learned dynamical layer from becoming spectrally invisible or phase-ambiguous.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Dynamical phase retrieval for Schr{ö}dinger evolution on finite graphs arXiv:2607.26705
Unverified 2026

Pair-Hydrodynamic Long-Memory State Space

Augment a stable diffusive state-space model with pair states formed from products of slow latent modes. Single modes represent ordinary long-wavelength diffusion, while pair modes represent the interacting hydrodynamic operators responsible for late-time tails in quartic observables. Use the pair states only for selected readout channels or a low-rank subset of mode pairs, preserving near-linear inference cost.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Interacting hydrodynamic modes in spinless fermions with dephasing noise arXiv:2607.25938
Unverified 2026

Resonant-Mode Observability Regularizer

For a learned recurrent or state-space model, estimate leading Koopman or transfer-operator modes and force their evaluations on a small set of latent states to be linearly independent. This transfers the paper's generic invertibility construction and discourages duplicated, weakly observable, or spectrally collapsed dynamical modes, potentially improving long-horizon prediction and interpretability.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Properties of resonant states for generic smooth expanding maps arXiv:2607.25686
Unverified 2026

Degenerate Invariant-Manifold RNN

Replace an unconstrained recurrent state update by a locally parameterized invariant manifold h equals K of z, where the latent dynamics z at the next step equal R of z and preserve slow modes near a degenerate fixed point. Train the embedding and reduced map jointly with an invariance residual, while a weighted lattice norm discourages perturbations in distant channels or spatial sites from growing.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Degenerate fixed points of maps in Banach spaces and lattices with decay and their invariant manifolds arXiv:2607.25577
Unverified 2026

Decomposed Mean-Field State Layer

Replace a single hidden state or a finite-order covariance/cumulant closure by an ensemble of independently propagated mean-field particles. The network output is reconstructed from particle averages, allowing bimodal and strongly non-Gaussian hidden-state distributions without explicitly evolving third- and higher-order tensors.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Don't truncate, decompose: mean-field dynamics of long-range quantum systems from strongly correlated states arXiv:2607.25434
Unverified 2026

Multiplicative Log-Time Update Clock

Use a multiplicative renewal clock to decide when a neural module is updated, rather than updating at every wall-clock tick or using a fixed iteration schedule. The resulting computation allocates many updates early and increasingly long intervals between later updates, while preserving a tunable stochastic distribution of update times; this is intended for anytime recurrent refinement, continual learning, or adaptive inference where late updates have diminishing marginal value.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Logarithmic Aging Diffusion from a Multiplicative Event Clock: Rare Event Statistics, Ultraslow Transport, and Ensemble-Time Inequivalence arXiv:2607.25374
Unverified 2026

Backward-error penalty for learned latent dynamics

Train a learned latent transition not merely to fit one-step data, but to require only a small operator correction before its selected spectral modes become exact eigenmodes. The correction is a measurable backward error, so the regularizer penalizes models whose apparent eigenstructure is highly sensitive to noise or finite-sample error. At inference time, the correction norm can trigger conservative rollout or mode suppression.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD arXiv:2607.25086
Unverified 2026

Low-Order Robust Functional Observer

Attach a small dynamical observer to a neural ODE, RNN, or state-space model and make it estimate only a task-relevant functional of the hidden state, such as logits, value features, or control-relevant projections. Use an incremental quadratic constraint and a bounded-real penalty to make the observer robust to hidden-state nonlinearities and input disturbances, instead of reconstructing the full latent state.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Functional H_infinity Filtering for Descriptor Systems with Incrementally Quadratic Nonlinearities under Disturbances arXiv:2607.25000
Unverified 2026

Diffusive fast-slow recurrent block

Replace a standard recurrent update with a slow-fast oscillator whose fast hidden state is coupled across feature channels by a graph-Laplacian diffusion term. The slow-fast structure permits sharp transient transitions, while diffusion suppresses unstable disagreement modes and should make long unrolled computation less sensitive to initialization and perturbations.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Diffusion stabilises time-periodic solutions in conservation laws coupled to a relaxation oscillator arXiv:2607.24994
Unverified 2026

Period-Resolvent Recurrent Layer

Construct a recurrent or graph-neural layer on a finite state space with a known bijection T, such as a modular cat map, and use the diagonal resolvent gain (1 − α^kx)^−1 as a state-dependent self-return or memory coefficient. States on short periodic orbits receive larger amplification, while long-period states receive weaker amplification, producing deterministic localization without learned disorder.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Arithmetic Landscape Functions of a Discrete Cat Map arXiv:2607.24857
Unverified 2026

Slow-Mode Adaptive Memory Gate

Use the spectral time constant of a memory operator to decide when a sequence layer should retain state, refresh it, or bypass expensive long-memory computation. A mode with eigenvalue near one is treated as valuable long memory, while unstable modes are suppressed, yielding an adaptive-computation mechanism driven by operator dynamics rather than token magnitude alone.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Memory operator ensembles indicate proximity to criticality in simulated AMOC transitions arXiv:2607.24310
Unverified 2026

RG-Decaying Rotational Residual Blocks

Construct a residual network with two coupled feature streams and deliberately non-reciprocal cross-stream interactions represented by a skew-symmetric coupling matrix. Decay the coupling strength with depth according to the RG picture of an irrelevant perturbation, allowing early layers to exploit rotational mixing while forcing deep layers toward reciprocal equilibrium-like dynamics. This should preserve transient expressivity without producing depth-dependent amplification or oscillatory…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Non-Reciprocal yet Equilibrium Critical Dynamics arXiv:2607.24252
Unverified 2026

Full-block IQC certificates for stable RNNs

Model an RNN as a linear state-space system in feedback with its slope-restricted activation, then search for a finite-horizon IQC multiplier instead of relying only on a spectral-radius or OZF-style condition. Penalize or reject parameter settings for which the strict IQC/LMI certificate has insufficient margin, yielding a directly testable stability criterion for long unrolled sequences.

Useful6/10
Difficulty7/10
Novelty6/10
Paper: Existence of stable Lur'e systems for which the O'Shea-Zames-Falb stability test fails arXiv:2607.23599
Unverified 2026

Homomorphic Tempo Conditioning

Condition a temporal neural network on a tempo or dilation ratio through a homomorphism from multiplicative positive scales to additive latent shifts. A ratio composed from several scale changes then produces the sum of their learned effects, allowing interpolation and extrapolation to rates absent from training instead of using an independent embedding per rate.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Infinite Canons: Maximally Self-Similar Melodic Lines and Canons with Infinite Solutions arXiv:2607.23210
Unverified 2026

Signature Half-Space Change Detector

Replace a recurrent sequence classifier's unconstrained hidden-state alarm head with an online truncated-signature state and a first-hitting-time linear detector. The module summarizes local order information and cross-channel interactions while preserving exact compositional updates, making it suitable for long streaming sequences and early-exit decisions.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Quickest Detection with Rough Path Signatures arXiv:2607.22958
Unverified 2026

Resonant Normal-Form Optimizer

Add a controlled periodic phase to an optimizer, then use a near-identity normal-form transform to remove rapidly oscillating gradient components instead of allowing them to perturb parameters directly. The optimizer follows averaged drift for non-resonant frequencies but explicitly preserves Fourier components near resonance, where they can create a secular update.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Resonance in coupled nonlinear oscillators with decaying perturbations arXiv:2607.22464
Unverified 2026

Cameron–Martin Adversarial Training

Replace isotropic input or hidden-state adversarial noise with an adversary that chooses a whole perturbation path in the Gaussian process's Cameron–Martin space. Penalizing the perturbation by its quadratic RKHS energy produces a risk-sensitive objective that attacks temporally coherent failure modes while avoiding unrealistic independent per-token noise.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Small-Noise Analysis of Controlled Functional Differential Equations with Gaussian Noise arXiv:2607.22362
Unverified 2026

Contractive Slow-State Cross-Coupled Reservoir

Build an RNN from fast nonlinear units coupled through a spectrally contractive slow state. The fast component can generate rich transients, while the slow component has a provable absorbing radius because its linear recurrence contracts and its neural forcing is bounded. Cross-coupling strength is swept to detect the onset of expressive high-dimensional attractors without permitting state explosion.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On a cross coupling of Rulkov neural maps arXiv:2607.22318
Unverified 2026

Imaginary-Axis Gramian Compression for Neural SSMs

Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: A New Low-Rank Cholesky-Factor ADI Algorithm Allowing Shifts Anywhere in the Complex Plane with Applications to Data-Driven Model Reduction arXiv:2607.21969
Unverified 2026

Nonresonant Quasi-Periodic Output Averaging

Add a deterministic torus phase to a recurrent or state-space model and average predictions over a quasi-periodic phase orbit using a frequency-aware normalized window instead of a uniform average. The window is chosen to attenuate Fourier modes near the orbit frequencies, transferring the paper's cancellation mechanism to reduce coherent long-horizon oscillation and bias without requiring a highly smooth predictor.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Exponential convergence can happen in weighted Birkhoff averages via quasi-periodicity with arbitrary nonresonance and low regularity arXiv:2607.21950
Unverified 2026

Invariant-Measure Training Monitor

Represent the optimizer state or recurrent hidden state as an iterated map and estimate its natural invariant measure from a sliding-window occupation histogram or feature embedding. Use convergence of long-run observable averages and distances between successive empirical measures to detect whether training has entered a stable, periodic, or chaotic statistical regime, and optionally control the learning rate without forcing pointwise convergence.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Natural Invariant Measures for Chaotic Game Dynamics: Finding Order in Chaos arXiv:2607.21805
Unverified 2026

Task-Targeted Spectral Excitation for Dynamics Learning

When training a neural state-space model, SSM, or recurrent world model from trajectories, constrain the data-generation policy or augmentation process to satisfy both a Hankel-rank condition and a task-weighted frequency-coverage condition. The rank condition prevents unidentifiable dynamics, while the frequency condition concentrates samples at frequencies that affect the target prediction horizon, tracking objective, or closed-loop controller instead of merely producing broadband-looking…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: When Persistency is not Exciting in Data-Driven Predictive Control arXiv:2607.21280
Unverified 2026

Phase-windowed synchronization layer

Augment each recurrent channel, feature group, or state-space stream with a latent phase oscillator and allow cross-stream coupling only when the receiving oscillator lies inside a learned or fixed phase window. The window suppresses destructive mixing outside the relevant dynamical regime while retaining Kuramoto-style attraction during the active interval, potentially improving long-horizon coherence without forcing all hidden states to synchronize continuously.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Kuramoto phase model to explore the synchronisation of a network of circadian clocks arXiv:2607.21214