Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Use the disorder-controlled stability boundary as a training schedule. Start with strong damping so optimization is well behaved, then reduce the damping margin toward zero to create long-lived oscillatory state memory after the network has learned useful representations.
Replace a Markovian recurrent update with an MPS-valued temporal influence state that couples adjacent pairs of memory sites, mimicking the paper's CDU3 two-column construction. The hidden state retains structured correlations across multiple past time steps while computation remains linear in sequence length and polynomial in the bond dimension, rather than exponential in the memory horizon.
Replace a uniformly stepped recurrent or state-space transition with propagation measured in an effective clock that may pause on intervals and make finite jumps at events. Use an implicit Stieltjes-Euler residual for every interval and event, then differentiate that exact residual with a reverse discrete adjoint. This should provide stable long inactive periods, exact scheduled resets, and fewer computational steps than approximating instantaneous events with many tiny chronological-time steps.
Replace independently restarted proximal-gradient or quasi-Newton solves for a composite neural objective with a curvature-recycling Douglas–Rachford loop. The previous proximal state, residual, and limited-memory BFGS curvature pairs are transported to the next proximal center, reducing expensive loss and gradient evaluations while retaining the cheap nonsmooth proximal operation.
Replace star-shaped parameter synchronization with a rooted-tree primal-dual optimizer in which each worker owns a parameter block and communicates only with its parent and children. Dual updates performed at a node are explicitly redistributed as child correction messages, preventing stale-consensus errors caused by level-synchronous execution.
Insert a slow routing state and an intermediate hysteresis variable between a neural memory and its next-state selector. The hysteresis prevents small prediction fluctuations from repeatedly changing the active attractor, while the slower router learns transition probabilities independently of the attractor parameters.
Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.
Use the q-fractional characteristic equation as an online trust-region controller for recurrent gain or residual-memory strength. Instead of allowing the recurrent Jacobian to cross the unit-circle boundary, estimate the dominant characteristic root and rescale the feedback gain whenever it approaches modulus one.
Replace a conventional feed-forward block with a sparse temporal graph whose hidden units are shared across many computation paths. Each arriving message updates a shared accumulator, applies a nonlinear response, and schedules delayed messages to downstream neurons; constructive or destructive interaction emerges when multiple paths visit the same unit.
Build a delayed recurrent layer whose state update contains explicit taps at lags k tau, and monitor whether its linearized dynamics support periodic or antiperiodic modes over a window of length m tau. Use the smallest singular value of the corresponding periodic-boundary residual as a bifurcation margin: values near zero indicate that a new oscillatory memory mode is being created or destroyed. The margin can be used either as a diagnostic or as a regularizer that keeps training away from…
Constrain each member of a wide recurrent or neural-ODE population to use the same time-dependent vector field whose spatial components generate a finite-dimensional Lie algebra. Store m fundamental trajectories and one fixed invariant label per node, then reconstruct every node state with the Lie-Scheffers superposition map instead of integrating all n states independently. The resulting layer has an exact md-dimensional dynamical core and should preserve the full network trajectory up to…
Replace an unconstrained recurrent matrix by a structured asymmetric circulant coupling whose Fourier modes have analytically known complex eigenvalues. A selected nonzero mode becomes a rotating attractor, providing a phase-coded recurrent state that can preserve information through oscillatory dynamics without requiring the optimizer to discover a stable spectral structure from scratch. A weak input projection and optional mode-selection loss can use the attractor as a nonlinear memory…
Represent an intermediate hidden-state matrix as a low-rank factorization and replace a dense residual update by a repeated contractive step followed by hard SVD truncation. Increase or decrease the rank automatically so that the discarded Frobenius-norm tail is below a specified tolerance, providing an explicit accuracy-memory tradeoff instead of fixing the rank in advance.
Construct each hierarchy transition as a coarse-graining map together with a local decoder that recovers the retained local representation. This creates a reversible-information path for features needed by later computation while still allowing high-frequency or locally irrelevant components to be discarded.
Add a recurrent associative matrix to each selected transformer layer so recent key-value relationships can be retrieved without retaining every past token or performing gradient updates. The matrix uses input-dependent retention and write gates, but retrieval is always performed from the pre-write state, preventing the current target from leaking into its own prediction. Frobenius-norm clipping makes the recurrent memory bounded and provides a direct stability control.
Impose a screened pair-correlation dynamics on stochastic neural replicas so that correlation fluctuations relax locally instead of propagating across the entire representation. The key control knob is a learned or scheduled relaxation rate \(\mu_{FB}\), which predicts a measurable correlation length \(\xi_{FB}=\sqrt{D_{eff}/\mu_{FB}}\). This can be used as a locality regularizer for token representations, diffusion trajectories, or recurrent hidden states.
Represent an iterative neural computation as a controlled dynamical system and learn sparse residual corrections that are active only for a finite prefix of iterations. Estimate local stable and anti-stable subspaces of the hidden-state Jacobian, increase the correction horizon only while the anti-stable component exceeds a tolerance, and force later controls to zero. This produces adaptive-depth inference with a quantitative stopping criterion.
Replace a learned dense recurrent transition with a truncated lowest-weight \(\mathrm{su}(1,1)\) ladder acting on hidden coordinates indexed by \(n=0,\ldots,N-1\). The ladder coefficients create a nonuniform, analytically specified coupling that grows with state index, while a negative \(J_0\) term supplies controllable dissipation and the skew combination \(J_+-J_-\) supplies conservative mixing.
Replace an unconstrained message-passing or recurrent propagation matrix by a directed-edge operator with non-backtracking connectivity and orientation-dependent turning phases, inspired by the Kac–Ward construction. During training, monitor and control the zero-momentum spectral gap of \(\mathcal A(0)=I-K(0)\), keeping the model near but on the stable side of the critical surface to obtain long memory without uncontrolled amplification.
Replace a generic optimizer over every discretized hidden state in a neural ODE or state-space model with a condensed reduced-space solve. At each outer Gauss-Newton or sequential-convex-programming iteration, linearize the neural dynamics, recursively eliminate all intermediate state increments, and apply projected primal-dual gradient updates to the remaining model parameters, controls, and terminal variables. This should be most useful when a model is trained with hard terminal targets…
Add a bounded routing state to an RNN, state-space model, or mixture-of-experts layer, with several neutral fixed points representing persistent modes. The state moves between modes when far from a fixed point but escapes each mode only polynomially when close to it, creating controllable long memory without setting a linear eigenvalue arbitrarily close to one. A temperature parameter selects between an entropy-rich phase using many modes and a low-entropy phase concentrated near one preferred…
Replace an unconstrained recurrent transition with a second-order resonant state whose restoring matrix is full-rank but whose damping is low-rank. The low-rank damping creates a small set of rapidly controlled bright modes and a large dark subspace with long memory, while a small optional damping term prevents numerical drift in completely dark modes.
Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.
Replace ordinary reverse-mode differentiation through a long sequence of stiff relaxation updates with a projected adjoint that separates slow conserved features from rapidly relaxing residual features. The neural layer can use large outer time steps even when its internal relaxation time is very small, while reconstructing only the microscopic gradient component required by the preceding layer.