ML: Rnn

Machine-learning ideas tagged Rnn in the ML taxonomy of the Math2NN corpus.

594 ideas found

Unverified 2026

Full-block IQC certificates for stable RNNs

Model an RNN as a linear state-space system in feedback with its slope-restricted activation, then search for a finite-horizon IQC multiplier instead of relying only on a spectral-radius or OZF-style condition. Penalize or reject parameter settings for which the strict IQC/LMI certificate has insufficient margin, yielding a directly testable stability criterion for long unrolled sequences.

Useful6/10
Difficulty7/10
Novelty6/10
Paper: Existence of stable Lur'e systems for which the O'Shea-Zames-Falb stability test fails arXiv:2607.23599
Unverified 2026

Homomorphic Tempo Conditioning

Condition a temporal neural network on a tempo or dilation ratio through a homomorphism from multiplicative positive scales to additive latent shifts. A ratio composed from several scale changes then produces the sum of their learned effects, allowing interpolation and extrapolation to rates absent from training instead of using an independent embedding per rate.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Infinite Canons: Maximally Self-Similar Melodic Lines and Canons with Infinite Solutions arXiv:2607.23210
Unverified 2026

Signature Half-Space Change Detector

Replace a recurrent sequence classifier's unconstrained hidden-state alarm head with an online truncated-signature state and a first-hitting-time linear detector. The module summarizes local order information and cross-channel interactions while preserving exact compositional updates, making it suitable for long streaming sequences and early-exit decisions.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Quickest Detection with Rough Path Signatures arXiv:2607.22958
Unverified 2026

Resonant Normal-Form Optimizer

Add a controlled periodic phase to an optimizer, then use a near-identity normal-form transform to remove rapidly oscillating gradient components instead of allowing them to perturb parameters directly. The optimizer follows averaged drift for non-resonant frequencies but explicitly preserves Fourier components near resonance, where they can create a secular update.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Resonance in coupled nonlinear oscillators with decaying perturbations arXiv:2607.22464
Unverified 2026

Cameron–Martin Adversarial Training

Replace isotropic input or hidden-state adversarial noise with an adversary that chooses a whole perturbation path in the Gaussian process's Cameron–Martin space. Penalizing the perturbation by its quadratic RKHS energy produces a risk-sensitive objective that attacks temporally coherent failure modes while avoiding unrealistic independent per-token noise.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Small-Noise Analysis of Controlled Functional Differential Equations with Gaussian Noise arXiv:2607.22362
Unverified 2026

Contractive Slow-State Cross-Coupled Reservoir

Build an RNN from fast nonlinear units coupled through a spectrally contractive slow state. The fast component can generate rich transients, while the slow component has a provable absorbing radius because its linear recurrence contracts and its neural forcing is bounded. Cross-coupling strength is swept to detect the onset of expressive high-dimensional attractors without permitting state explosion.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On a cross coupling of Rulkov neural maps arXiv:2607.22318
Unverified 2026

Imaginary-Axis Gramian Compression for Neural SSMs

Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: A New Low-Rank Cholesky-Factor ADI Algorithm Allowing Shifts Anywhere in the Complex Plane with Applications to Data-Driven Model Reduction arXiv:2607.21969
Unverified 2026

Nonresonant Quasi-Periodic Output Averaging

Add a deterministic torus phase to a recurrent or state-space model and average predictions over a quasi-periodic phase orbit using a frequency-aware normalized window instead of a uniform average. The window is chosen to attenuate Fourier modes near the orbit frequencies, transferring the paper's cancellation mechanism to reduce coherent long-horizon oscillation and bias without requiring a highly smooth predictor.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Exponential convergence can happen in weighted Birkhoff averages via quasi-periodicity with arbitrary nonresonance and low regularity arXiv:2607.21950
Unverified 2026

Invariant-Measure Training Monitor

Represent the optimizer state or recurrent hidden state as an iterated map and estimate its natural invariant measure from a sliding-window occupation histogram or feature embedding. Use convergence of long-run observable averages and distances between successive empirical measures to detect whether training has entered a stable, periodic, or chaotic statistical regime, and optionally control the learning rate without forcing pointwise convergence.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Natural Invariant Measures for Chaotic Game Dynamics: Finding Order in Chaos arXiv:2607.21805
Unverified 2026

Task-Targeted Spectral Excitation for Dynamics Learning

When training a neural state-space model, SSM, or recurrent world model from trajectories, constrain the data-generation policy or augmentation process to satisfy both a Hankel-rank condition and a task-weighted frequency-coverage condition. The rank condition prevents unidentifiable dynamics, while the frequency condition concentrates samples at frequencies that affect the target prediction horizon, tracking objective, or closed-loop controller instead of merely producing broadband-looking…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: When Persistency is not Exciting in Data-Driven Predictive Control arXiv:2607.21280
Unverified 2026

Phase-windowed synchronization layer

Augment each recurrent channel, feature group, or state-space stream with a latent phase oscillator and allow cross-stream coupling only when the receiving oscillator lies inside a learned or fixed phase window. The window suppresses destructive mixing outside the relevant dynamical regime while retaining Kuramoto-style attraction during the active interval, potentially improving long-horizon coherence without forcing all hidden states to synchronize continuously.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Kuramoto phase model to explore the synchronisation of a network of circadian clocks arXiv:2607.21214
Unverified 2026

Boundary-Coupled Spectral Memory Layer

Replace a generic recurrent transition with a finite spectral approximation of the paper's augmented generator: one state block represents ordinary latent dynamics and another represents delayed or refractory history. Inject the input through two learned channels, analogous to bulk forcing and boundary-condition forcing, so the model can represent abrupt events and delayed consequences without requiring a large delay buffer. Parameterize selected mode pairs as stable real Jordan blocks or…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral theory for population density dynamics of spiking neurons with refractoriness arXiv:2607.20699
Unverified 2026

Constraint-Free Skew Coupling

Compose independently parameterized neural dynamical modules through power-preserving skew coupling instead of equality penalties or projected constraints. This creates a modular graph or world model in which information exchanged between modules is antisymmetric, so internal coupling cannot create or destroy total latent energy.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Mixed finite element discretization of intrinsic geometrically exact beams for explicit multibody dynamics arXiv:2607.20245
Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
Unverified 2026

Cancellation-Aware Tree Neural CDE Step

Implement a neural controlled differential equation update using a truncated planar-binary-tree expansion rather than a first-order Euler step. Select the truncation order from driver regularity and the observed magnitudes of elementary differentials, while using a cancellation-aware remainder monitor to avoid computing unnecessarily high-order terms.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Remainders of generalised Taylor expansions and a priori bounds for rough differential equations arXiv:2607.18635
Unverified 2026

Equal-amplitude synchronized oscillator modes

Use multiple oscillator modes with weak phase coupling and regularize their active amplitudes toward a common squared amplitude. This transfers the paper's conclusion that coupled nonzero modes satisfy $A_j^2=A_k^2$ or that a mode collapses to zero, producing a controllable mixture of synchronized persistent modes and suppressed modes.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Coupled Van der Pol Networks arXiv:2607.18337
Unverified 2026

Nested-transport depth consistency

Apply the paper's nested coupling between path distributions at two Krasnosel'skii–Mann depths to an iterative neural block. Penalize discrepancies between intermediate representations using the coupling mass, so that the short unroll learns to approximate the long unroll while preserving the block's actual computational-path geometry. At inference, use the resulting coupled discrepancy as an early-exit criterion.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Krasnosel'skii-Mann iterations beyond asymptotics: a combinatorial analysis arXiv:2607.18121
Unverified 2026

Log-Fourier Normal-Form Recurrent Cell

Construct a second-order recurrent cell with an odd high-degree restoring force and lower-degree state-dependent velocity feedback, while representing time-dependent coefficients as a finite Fourier series. At each training or inference window, retain and normalize only Fourier modes below K = c_* log A, where A is the current hidden-state amplitude; apply bounded corrections to nonresonant low modes and leave the analytically small high-frequency tail untouched. The predicted benefit is…

Useful6/10
Difficulty7/10
Novelty8/10
Paper: Lagrange Stability for Reversible Duffing Equations with Quasi-Periodic Coefficients arXiv:2607.17068
Unverified 2026

C1 Homogeneous Lyapunov Critic

For a neural ODE or recurrent state update, learn a positive-definite degree-two homogeneous Lyapunov function that is only C1, rather than restricting the certificate to polynomials or analytic neural networks. Parameterize its angular dependence with a positive spline or softplus mixture, and train it to decrease along the learned vector field; this can certify stable dynamics that polynomial Lyapunov searches systematically miss.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: A Globally Asymptotically Stable Planar Homogeneous Polynomial Vector Field With No Polynomial Lyapunov Function arXiv:2607.16171
Unverified 2026

Laplace-Margin Regularized Depression RNN

Augment a recurrent or state-space layer with a bounded synaptic-depression variable that multiplicatively reduces recurrent transmission after activity. During training, estimate the layer's impulse-response transform and penalize characteristic roots approaching the unstable half-plane. This directly targets slow oscillations and exploding recurrent feedback rather than relying only on gradient clipping.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On large networks of integrate-and-fire neurons with short-term synaptic plasticity arXiv:2607.16017
Unverified 2026

Singular-perturbation continuation schedule

Use the paper's fast-layer/reduced-problem decomposition as a training schedule: first optimize a cheap reduced neural dynamics on the critical manifold, then gradually restore the fast dynamics by increasing the stiffness parameter. This provides a continuation path from an easy slow problem to the intended recurrent or implicit model and supplies a concrete stopping criterion based on normal-hyperbolicity loss.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Constructing far-from-equilibrium patterns in a cross-diffusion vegetation-autotoxicity model arXiv:2607.15692
Unverified 2026

Fold-aware fast-slow neural state layer

Replace a single recurrent or neural-ODE state update by a fast subsystem for the rapidly relaxing state and a slow subsystem for context, memory, or parameters. Constrain the learned algebraic critical manifold to remain normally hyperbolic during ordinary operation, while treating its folds as explicit, detectable transition surfaces that can generate controlled regime changes rather than numerical blow-up.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Constructing far-from-equilibrium patterns in a cross-diffusion vegetation-autotoxicity model arXiv:2607.15692
Unverified 2026

Periodic-block recurrent dynamics

Replace a generic recurrent transition by an exactly periodic unitary base transition plus a learnable weak Hermitian perturbation. The resulting \(\tau\)-step macro-dynamics approximates a continuous-time unitary flow, allowing the model to preserve signal norms while learning slowly varying long-range transformations.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Robustness of periodicity in Grover walks under a magnetic vector potential arXiv:2607.14797
Unverified 2026

Delay-Resonance Monitor for Oscillatory Hidden States

Augment a recurrent or state-space neural network with an explicit delayed hidden-state channel and monitor the linearized delay spectrum around the zero or operating-point state. Use the paper's antiperiodic resonance equations to predict when oscillatory hidden modes should appear, then either avoid those parameter regions for stable sequence prediction or deliberately target them for periodic-memory tasks.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Bifurcations of periodic and antiperiodic orbits near an equilibrium in autonomous differential delay systems with one or two delays arXiv:2607.14533