Unverified 2026

Threshold-Projection Recurrent Memory

Implementation & benchmark of arXiv:2607.14321 — Accounting for Hysteresis and Eddy Currents in Finite Element Simulations of Ferromagnetic Laminated Cores using a Recurrent Neural Network

Usefulness6/10

Difficulty4/10

Novelty7/10

Architecture Stability Accuracy Sample efficiency Rnn Ssm World model Geometry Dynamical systems Optimization

Source paper: Accounting for Hysteresis and Eddy Currents in Finite Element Simulations of Ferromagnetic Laminated Cores using a Recurrent Neural Network arXiv:2607.14321 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace or augment a recurrent cell with multiple hysteresis memory branches whose states remain unchanged while the input stays within a branch-specific radius, then move toward the current input only when that radius is exceeded. The resulting cell has explicit persistence and bounded state changes, giving it an inductive bias for temporal hysteresis and reducing the need for the network to learn long-term memory behavior from scratch.

Formulas

$$\mathbf{h}_{\mathrm{rev}}^{n}=\begin{cases}\mathbf{h}_{\mathrm{rev},\mathrm{p}}^{n}&\text{if }\left\lVert\mathbf{h}-\mathbf{h}_{\mathrm{rev},\mathrm{p}}^{n}\right\rVert_2\leq\kappa^{n},\\[2mm]\mathbf{h}-\kappa^{n}\frac{\mathbf{h}-\mathbf{h}_{\mathrm{rev},\mathrm{p}}^{n}}{\left\lVert\mathbf{h}-\mathbf{h}_{\mathrm{rev},\mathrm{p}}^{n}\right\rVert_2}&\text{otherwise,}\end{cases}$$

$$\mathbf{h}_{\mathrm{rev}}=\sum_{n=1}^{N}w^{n}\mathbf{h}_{\mathrm{rev}}^{n},\qquad w^{n}\geq0,\qquad\sum_{n=1}^{N}w^{n}=1,$$

$$\mathbf{h}_{\mathrm{rev}}^{n}=\Pi_{B(\mathbf h,\kappa^n)}\!\left(\mathbf{h}_{\mathrm{rev},p}^{n}\right),\qquad B(\mathbf h,\kappa^n)=\{\mathbf z:\|\mathbf z-\mathbf h\|_2\leq\kappa^n\},$$

$$\mathbf y_t=\operatorname{MLP}\!\left([\mathbf x_t,\,\mathbf m_t,\,\mathbf m_t-\mathbf m_{t-1}]\right),\qquad \mathbf m_t=\sum_{n=1}^{N}w_n\,\Pi_{B(\mathbf u_t,\kappa_n)}(\mathbf m_{t-1}^{n}),\quad \mathbf u_t=W_x\mathbf x_t+b_x,$$

Mathematical statement

The paper's explicit hysteresis update is a projection rule for each branch n. Given current input vector $\mathbf h\in\mathbb R^d$, previous branch state $\mathbf h_{\mathrm{rev},p}^{n}\in\mathbb R^d$, and threshold radius $\kappa^n>0$, the new state is $\mathbf h_{\mathrm{rev}}^{n}=\mathbf h_{\mathrm{rev},p}^{n}$ when $\|\mathbf h-\mathbf h_{\mathrm{rev},p}^{n}\|_2\leq\kappa^n$, otherwise it is $\mathbf h-\kappa^n(\mathbf h-\mathbf h_{\mathrm{rev},p}^{n})/\|\mathbf h-\mathbf h_{\mathrm{rev},p}^{n}\|_2$. Equivalently, $\mathbf h_{\mathrm{rev}}^{n}$ is the Euclidean projection of the previous state onto the closed ball centered at $\mathbf h$ with radius $\kappa^n$: $\Pi_{B(\mathbf h,\kappa^n)}(\mathbf h_{\mathrm{rev},p}^{n})$. The paper aggregates branches as $\mathbf h_{\mathrm{rev}}=\sum_{n=1}^{N}w^n\mathbf h_{\mathrm{rev}}^n$, where $w^n\geq0$ and $\sum_n w^n=1$. The projection bounds every branch's distance from the current input by $\kappa^n$, while convex aggregation keeps the combined state inside the corresponding weighted geometric envelope. For neural networks, use $\kappa^n=\operatorname{softplus}(\rho^n)+\epsilon$, $w^n=\operatorname{softmax}(a)_n$, and optionally learn an input projection before applying the rule.

Implementation notes

(1) Integration point: implement a PyTorch module HysteresisMemory for sequential data. At timestep t, map the input tensor x_t with shape [batch,d_in] to u_t = W_x @ x_t + b_x with shape [batch,d]; maintain N state tensors m_prev[n] of shape [batch,d]. This can replace a GRU hidden-state update, sit before an SSM/readout layer, or run in parallel to a standard recurrent layer. (2) Pseudocode: initialize every m_prev[n] to zero; compute kappa = softplus(rho)+1e-4 and w = softmax(a); for each timestep and branch compute delta = u_t - m_prev[n], dist = sqrt(sum(delta**2, dim=-1, keepdim=True)+1e-8), and candidate = u_t - kappa[n]*delta/dist; set m_new[n] = where(dist <= kappa[n], m_prev[n], candidate); aggregate m_t = sum_n w[n]*m_new[n]; predict with head(concat(x_t,m_t,m_t-m_prev_aggregate)); then assign m_prev = m_new. The displayed paper formula is exactly the conditional update and radial projection. (3) Mathematics versus estimation: the projection rule, Euclidean-ball geometry, and normalized branch weights are fixed by the construction. Learn rho, a, W_x, and the prediction head by backpropagation. Estimate a useful initialization for each radius from displacement quantiles of u_t-u_{t-1}, for example initialize kappa[n] to quantiles from 0.1 to 0.9. A smooth training variant uses g=sigmoid((dist-kappa)/tau) and m_new=(1-g)*m_prev+g*candidate; anneal tau and finally evaluate the hard rule. (4) First cheap experiment: train on Moving-MNIST frame prediction or an electricity/load forecasting dataset using a one-layer GRU with matched hidden size and approximate FLOPs as baseline. Compare GRU, a vanilla leaky integrator, and this module for N in {4,8,16}. Measure validation MSE, long-horizon error, performance under repeated or reversed input segments, gradient norms over 1,000-step sequences, and inference throughput. Success means lower long-horizon error or a smaller hidden state at equal error, with fewer exploding or vanishing-gradient failures and stable predictions when the input reverses direction.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.