# Thresholded Hidden-State Restart Gate

- ID: 2105
- Canonical URL: https://synthcore.org/idea/2105/thresholded-hidden-state-restart-gate
- API JSON: https://synthcore.org/api/idea/2105.json
- API Markdown: https://synthcore.org/api/idea/2105.md
- Verification status: unverified
- Source: [arXiv:2608.10936](https://arxiv.org/abs/2608.10936)
- Category: dynamics
- Solves: stability, accuracy, scalability
- ML areas: rnn, ssm, training-dynamics
- Math tags: probability, dynamical-systems, optimization
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Replace an unconstrained recurrent reset gate with a threshold policy over hidden-state age and a scalar degradation score. The model continues its recurrence while the estimated cost of retaining the state is below the cost of restarting, then resets and reinitializes the state when the threshold is crossed. This should reduce long-horizon hidden-state drift while using only one scalar gate per sequence position.

## Mathematical statement

The paper's restart kernel is \(\hat P(dx'\mid x,a)=P(dx'\mid x)\) for continuation \(a=n\), and \(\hat P(dx'\mid x,a)=\nu(dx')\) for restart \(a=r\), where \(x\) is the hidden state, \(P\) is its natural transition kernel, and \(\nu\) is the restart distribution. Restart also reveals the new state while continuation reveals no state; this motivates the sufficient statistic \((x_0,t)\), with \(x_0\) the last reset state and \(t\) the elapsed time. For a discounted objective with per-step cost \(c(x_0,t,a)\), discount \(\gamma\in(0,1)\), and value \(V\), the continuation and restart action values are \(Q_n(x_0,t)=c(x_0,t,n)+\gamma V(x_0,t+1)\) and \(Q_r(x_0,t)=c(x_0,t,r)+\gamma\mathbb E_{x'\sim\nu}[V(x',0)]\). The Bellman equation is \(V(x_0,t)=\min\{Q_n(x_0,t),Q_r(x_0,t)\}\). Under the paper's one-step cost-deterioration condition, the optimal action has threshold form: continue for \(t<\tau(x_0)\) and restart for \(t\geq\tau(x_0)\). If the state is partially ordered and the transition kernel is stochastically monotone, the threshold is monotone in \(x_0\). In the neural adaptation, \(x_0\) is represented by the post-reset hidden state, \(t\) is hidden-state age, and a learned scalar \(d_t\) estimates deterioration; the practical gate is \(r_t=\mathbf 1[t\geq\tau(d_t)]\).

## Key formulas

- $$\hat P(dx^{\prime}|x,a)=\begin{cases}P(dx^{\prime}|x)&\text{if }a=n,\\\nu(dx^{\prime})&\text{if }a=r,\end{cases}$$
- $$Q_n(x_0,t)=c(x_0,t,n)+\gamma V(x_0,t+1),\qquad Q_r(x_0,t)=c(x_0,t,r)+\gamma\mathbb E_{x'\sim\nu}[V(x',0)]$$
- $$V(x_0,t)=\min\{Q_n(x_0,t),Q_r(x_0,t)\},\qquad \pi^*(x_0,t)=\begin{cases}n,&t<\tau(x_0),\\r,&t\geq\tau(x_0).\end{cases}$$
- $$h_t=(1-r_t)\,F_\theta(h_{t-1},u_t)+r_t\,h_{\mathrm{init}},\qquad r_t=\mathbf 1\!\left[t\geq\tau_\phi(d_t)\right],\quad d_t=\operatorname{softplus}\!\bigl(w^\top[h_t^{\mathrm{raw}},u_t]-b\bigr)$$

## Implementation notes

Integrate the gate into a GRU, LSTM, or diagonal state-space model immediately after computing the ordinary candidate state \(h_t^{\mathrm{raw}}=F_\theta(h_{t-1},u_t)\). Maintain an integer age \(t\), reset it to zero whenever the gate fires, and use \(h_t=(1-r_t)h_t^{\mathrm{raw}}+r_t h_{\mathrm{init}}\), where \(h_{\mathrm{init}}\) is either a learned vector or a small encoder of the current input. Compute a scalar deterioration score \(d_t=\operatorname{softplus}(w^\top[h_t^{\mathrm{raw}},u_t]-b)\). Start with a monotone threshold \(\tau(d)=\operatorname{clip}(\tau_0-\alpha d,1,T_{\max})\), with \(\alpha\geq0\), so larger estimated deterioration causes earlier restart. During training use the straight-through estimator: the forward gate is \(r_t=\mathbf1[t\geq\tau(d_t)]\), while the backward pass uses the sigmoid surrogate \(\tilde r_t=\sigma((t-\tau(d_t))/s)\). Add a restart penalty \(\lambda_r r_t\) and train with the task loss plus this penalty; anneal temperature \(s\) from 1 to 0.1. The paper's mathematical contribution is the constrained threshold policy and restart-versus-continuation Bellman comparison; estimate the deterioration score and threshold parameters empirically. First test on a 1-2 layer GRU and a diagonal SSM on WikiText-2 character prediction and Long Range Arena copy/addition tasks, comparing against the same model with its standard learned reset gate and fixed periodic resets. Measure validation loss, gradient norm, hidden-state norm drift, number of resets, and throughput. Success is lower long-context loss and fewer exploding or vanishing-gradient events at comparable reset count, or equal loss with substantially fewer recurrent state updates.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.