# Hysteretic Attractor Layer

- ID: 93
- Canonical URL: https://synthcore.org/idea/93/hysteretic-attractor-layer
- API JSON: https://synthcore.org/api/idea/93.json
- API Markdown: https://synthcore.org/api/idea/93.md
- Verification status: unverified
- Source: [arXiv:2608.23225](https://arxiv.org/abs/2608.23225)
- Category: dynamics
- Solves: stability, memory, accuracy
- ML areas: rnn, ssm, memory
- Math tags: dynamical-systems, combinatorics
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a binary recurrent layer's ordinary sign activation with a hysteretic update that retains each unit's previous state when its incoming weighted sum is inconclusive. This creates discrete attractors that are robust to small perturbations and can serve as persistent latent memories, iterative classifiers, or compact associative states.

## Mathematical statement

The paper defines binary states $x_i(t)\in\{-1,+1\}$, recurrent fields $y_i(t)=\sum_{j=1}^{N}w_{ij}x_j(t)$, and the update $x_i(t+1)=h(y_i(t))$. For threshold $Th>0$, the hysteresis map is $h(y_i(t))=+1$ if $y_i(t)\ge Th$, $h(y_i(t))=x_i(t)$ if $|y_i(t)|<Th$, and $h(y_i(t))=-1$ if $y_i(t)\le -Th$. The interval $(-Th,Th)$ is therefore a deadband in which the previous state is copied. A fixed point $z\in\{-1,+1\}^N$ satisfies $z_i=h(\sum_jw_{ij}z_j)$ for every $i$; its basin is the set of initial states whose iterates converge to $z$. The paper uses integer weights and half-integer thresholds $Th\in\{0.5,1.5,2.5,\ldots\}$, avoiding equality ambiguity because the fields are integer. The neural-network adaptation uses the exact discrete map in the forward pass and a differentiable straight-through surrogate in the backward pass.

## Key formulas

- $$x_i(t+1)=h(y_i(t)),\qquad y_i(t)=\sum_{j=1}^{N}w_{ij}x_j(t),\qquad x_i(t)\in\{-1,+1\}.$$
- $$h(y_i(t))=\begin{cases}+1,&y_i(t)\ge Th,\\x_i(t),&|y_i(t)|<Th,\\-1,&y_i(t)\le-Th,\end{cases}\qquad Th>0.$$
- $$z_i=h\!\left(\sum_{j=1}^{N}w_{ij}z_j\right),\quad i=1,\ldots,N.$$
- $$\tilde h_\tau(y,x)=\tanh(y/\tau)+\rho(Th-|y|)x,$$

## Implementation notes

(1) Integration point: place the module after an encoder or inside an RNN/SSM latent update. The encoder outputs $u\in\mathbb R^N$; initialize $x(0)=\operatorname{sign}(u)$ and run $K$ recurrent updates with learned matrix $W$. (2) Pseudocode: `x = sign(encoder(input)); for t in 1..K: y = W @ x; x = where(y >= Th, +1, where(y <= -Th, -1, x)); output = readout(x)`. In the forward pass use the exact hysteresis rule. In the backward pass use a straight-through estimator whose surrogate is $\tilde h_\tau(y,x)=\tanh(y/\tau)+\rho(Th-|y|)x$; stop gradients through the copied-state term if it causes instability, and anneal $\tau$ from 1 to 0.1. (3) Compute the recurrent field and threshold comparisons exactly; estimate gradients empirically through the surrogate. Constrain $W$ by spectral normalization or replace it with $W/\max(1,\|W\|_2)$ to reduce oscillations. (4) First experiment: train a 32-unit MNIST recurrent classifier for 10 update steps, comparing hysteretic activation against tanh and sign baselines at equal parameter count and FLOPs. Evaluate clean accuracy, accuracy after additive input noise, accuracy after random latent bit flips, convergence rate defined by $x(t+1)=x(t)$, and the number of state changes in the final three iterations. The hypothesis is improved robustness and more stable late-iteration predictions, with comparable clean accuracy.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.