ML: Rnn

Machine-learning ideas tagged Rnn in the ML taxonomy of the Math2NN corpus.

583 ideas found

Mechanism failed 2026

Subresonant Power-Law Memory Bank

Add a deterministic complex-valued state-space bank whose mode detunings become progressively smaller with mode index, Delta_n=c n^{-p}, while input couplings decay as B_n=b n^{-kappa}. For slowly varying or constant forcing, the summed state follows the paper's subresonant response and grows like t^{1-alpha}, providing controllable power-law memory with only O(N) recurrent state updates. This should improve long-context retention compared with a same-size unconstrained RNN or uniformly spaced…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Small Denominators and Subresonant Accumulation in Weakly Nonlinear Dispersive Dynamics arXiv:2607.01447
Mechanism works 2026

Conjugacy-Regularized Latent Dynamics

Replace orthogonal Procrustes alignment between two latent dynamical systems with a learned bijection h that makes their transitions commute: h(f(z)) approximately equals g(h(z)). Parameterize h as an invertible affine map or coupling flow, allowing the correspondence to be non-orthogonal while retaining an exact inverse. The same constraint can be applied over multiple rollout steps, encouraging two models to represent the same computation even when their latent coordinates differ…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Beyond DSA: Conjugacy-based Comparison of Dynamical Systems arXiv:2607.04493
Mechanism failed 2026

Canonical Tropical Segment State

Build a sequence or graph module whose state is a canonical set of affine tropical pieces rather than an opaque hidden vector. Compose consecutive segments by Minkowski addition of their lifted supports, merge alternatives by union followed by lower-hull reduction, and evaluate the resulting piecewise-linear function with a minimum.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: On the Representational Geometry of Dynamic Programs arXiv:2608.25034
Mechanism failed 2026

Lyapunov-gap regularization for recurrent dynamics

Regularize a recurrent or state-space model using finite-time Lyapunov exponents of its actual hidden-state transition products. Penalize collapsed adjacent exponents while also controlling the largest exponent, encouraging several useful state directions instead of one dominant direction or universal contraction.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Quantitative Furstenberg Theory for Large Random Matrices arXiv:2608.22543
Mechanism failed 2026

CFL-Optimized Palindromic Residual Block

Replace an explicit Euler residual update for a skew-coupled hidden state with a five-stage palindromic composition of exact shear maps. Use a=1/4, the unique real coefficient maximizing the analyzed spectral CFL interval, and adapt the step size from an estimate of the learned coupling operator's spectral norm.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Sharp CFL stability and temporal-dispersion optimization of symmetric splitting schemes for time-domain Maxwell equations arXiv:2608.22315
Mechanism failed 2026

Delay-Ring Replay Memory

Replace a directed sequence-memory chain with a circular recurrent state propagated by a learned delayed convolution. The same learned kernel can support forward and reverse replay because replay direction is a dynamical mode of the ring, rather than requiring plasticity to explicitly learn both forward and backward synapses.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Forward and reverse delay-driven hippocampal replay without symmetric plasticity arXiv:2608.21814
Unverified 2026

Almost-Commuting State Dynamics

Use two bounded self-adjoint transition operators in a recurrent or state-space block and penalize their normalized Hilbert--Schmidt commutator. When the penalty is small, the paper guarantees that the pair is close to exactly commuting operators, suggesting a controlled path to a shared eigenbasis and cheaper coordinate-wise dynamics. Add an optional numerical repair step that projects the learned pair toward a simultaneously diagonalizable pair.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: On almost commuting matrices with respect to the normalized Hilbert--Schmidt norm arXiv:2608.31000
Unverified 2026

Pressure-Controlled Neural IFS

Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Dimensions of surface repellers and attractors of non-linear planar IFSs arXiv:2608.30744
Unverified 2026

Annihilating Density-Transport Feature Mixer

Replace an ordinary local convolution or token-mixing block by two nonnegative feature populations A and B that diffuse and drift along the spatial or token axis, with transport rates increasing quadratically with local population and with an optional directional bias. Add a local A plus B to empty reaction so mutually conflicting feature mass is removed rather than merely averaged. The module should produce adaptive competition, and its isolated relaxation should exhibit a measurable t raised…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Effective Hardcore Exclusion Without Exclusion: Two-Species Pair Annihilation Revisited arXiv:2608.30630
Unverified 2026

Equilibrium-Gain Sentinel

Add a low-dimensional, trusted sentinel state to the optimizer or recurrent inference controller. The sentinel is driven by a secret probe and a protected gain, so unauthorized gain changes produce a predictable shift in its equilibrium even when the main neural dynamics remain numerically stable. Monitor the estimated equilibrium and trigger rollback or quarantine when the measured shift exceeds the expected noise envelope.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Exposing the Invisible: Detecting Stealthy Parameter-Based Cyber-Attacks on Inverter Synchronization Loops arXiv:2608.30574
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Reversible Two-Relaxation Recurrent Block

Replace the recurrent transition or state-space mixer with a reversible transport followed by complementary relaxation of symmetric and antisymmetric feature components. The construction preserves a weighted energy and damps both parity sectors, giving bounded long-horizon powers without requiring the learned transition matrix itself to be symmetric. A numerical-range ellipse can be used as a cheap training-time certificate against transient growth.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Mesh-Uniform Power Stability of Two-Relaxation-Time Vector Lattice Boltzmann Schemes with Reversible Boundaries arXiv:2608.30253
Unverified 2026

Composite-cycle interface search

Use exhaustive finite-state analysis as an architecture-search objective for coupled recurrent motifs. Instead of independently tuning a normalization gate and a WTA gate, enumerate their possible interfaces and select couplings that create group action only on a small joint image set, yielding controlled reversible subdynamics embedded in an otherwise dissipative system.

Useful6/10
Difficulty6/10
Novelty9/10
Paper: "More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks arXiv:2608.30231
Unverified 2026

Block-Hurwitz barrier for polynomial state-space layers

Constrain the learned coefficients of a high-order linear recurrent or state-space layer using the block Hurwitz matrix associated with its matrix characteristic polynomial. Penalize near-singular Hurwitz blocks and, for degrees two and three, optionally enforce positive leading Hurwitz determinants; use companion-matrix eigenvalues as the definitive stability check rather than trusting determinant positivity at degree four or above.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: On the Coefficients of Hurwitz-Type Matrix Polynomials arXiv:2608.30089
Unverified 2026

Row-balanced recurrent initialization

Initialize or regularize recurrent matrices so that each unit receives an approximately cancelling sum of positive and negative weights, while keeping the global variance and spectral radius fixed. Sweep a continuous balance parameter instead of imposing balance blindly, because the paper predicts qualitatively different behavior for saturating, sub-linear, and odd nonlinearities.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Local connectivity balance shapes population dynamics in random recurrent networks arXiv:2608.30008
Unverified 2026

Inertia-Constrained Kreiss RNN

Replace an unconstrained recurrent transition matrix with a J-selfadjoint matrix A, where J is a fixed diagonal signature matrix with only a small number of negative entries. Add a sampled Kreiss-resolvent penalty to suppress transient amplification while preserving the expressive dimension of the hidden state. The paper's bound predicts that worst finite-time amplification depends on the smaller inertia index rather than the full hidden dimension.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Inertia-Sensitive Kreiss Bounds for $J$-Selfadjoint Matrices arXiv:2608.29823
Unverified 2026

Screened Cross-Correlation Regularizer

Impose a screened pair-correlation dynamics on stochastic neural replicas so that correlation fluctuations relax locally instead of propagating across the entire representation. The key control knob is a learned or scheduled relaxation rate \(\mu_{FB}\), which predicts a measurable correlation length \(\xi_{FB}=\sqrt{D_{eff}/\mu_{FB}}\). This can be used as a locality regularizer for token representations, diffusion trajectories, or recurrent hidden states.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Branching stochastic mechanics. I. Clustering and connected correlations within a branching-process representation of the Schrödinger equation arXiv:2608.29807
Unverified 2026

Relative-Degree-Gated Passive Neural State Space

Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Relative-Degree Wall Restricts Passivity-Based Stability Analysis in Inverter-Dominant Grids arXiv:2608.29474
Unverified 2026

Laguerre-Optimal Positive Delay Filter

Replace an Erlang delay or exponential smoothing cascade in a recurrent or state-space layer by a positive rational kernel of the form \(\kappa(u)=C e^{-a u}p(u)^2\). Choose the degree-\(m\) polynomial by deleting the adjacent pair of Laguerre zeros with smallest relative gap from \(L_{m+2}\), then rescale the resulting density to unit mean. This preserves a nonnegative impulse response while reducing temporal jitter relative to Erlang filters.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Least Variability in a Polynomial-Square Class of Rational Kernels arXiv:2608.29143
Unverified 2026

Uniformly Mixing Nonstationary State-Space Network

Build a recurrent or state-space network with time-dependent transition parameters, but train it to forget perturbations at a common exponential rate across all admissible parameter schedules. The model should retain task-relevant long-term signals while suppressing dependence on arbitrary initial hidden states, reducing instability under changing inputs, curricula, or deployment-time dynamics.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Decay of correlations and normal approximation for nonstationary heterochaos baker maps arXiv:2608.29135
Unverified 2026

Cycle-Current Integrability Monitor

For a neural stochastic state-space model or discrete diffusion sampler, monitor whether learned transition logits admit a global scalar potential on the active latent manifold. Penalize residual cycle affinities in the conditional sector, but leave reset cycles unpenalized so the model can retain useful dissipative mixing. The distinctive prediction is a linear decrease of integrability error with residual cycle current and a quadratic decrease of entropy production near autonomous…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: When dissipative steady states admit thermodynamic occupation laws arXiv:2608.26621
Unverified 2026

Supercritical Hopf Latent Cell

Replace an unconstrained recurrent hidden-state channel with a two-dimensional oscillator constrained to the supercritical Hopf normal form. A learned control parameter can place the channel below threshold for decaying dynamics or above threshold for sustained periodic dynamics, while the cubic term bounds the amplitude and prevents recurrent-state explosion.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Minimal Thermodynamically Consistent Chemical Oscillator arXiv:2608.24200
Unverified 2026

Unstable-Manifold-Aware Ensemble Averaging

For a neural dynamical predictor, train or maintain several independently initialized models and aggregate their multi-step states using the signed displacement along the locally unstable forecast direction. The key mechanism is cancellation of opposite unstable-manifold errors: ordinary averaging should reduce this component at rate N^{-1/2} when errors are independent and centered, while robust aggregation should be activated when validation residuals show heavy tails or persistent bias.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Understanding the superiority of multi-model ensemble forecasts through reservoir computing arXiv:2608.20017
Unverified 2026

Product-Matched Spectral Trust Region

Use the paper's product-matched uniform cycle as a tractable spectral envelope for a cyclic recurrent or state-space layer. Instead of estimating the full nonnormal generator spectrum at every update, compute its forward and backward rate products and constrain each complex eigenmode to remain inside the corresponding comparison-cycle frequency bound.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Coarse-grained kinetic scale tightens thermodynamic spectral bounds of Markov cycles arXiv:2608.22934