# Elephant Adaptive Threshold

- ID: 2210
- Canonical URL: https://synthcore.org/idea/2210/elephant-adaptive-threshold
- API JSON: https://synthcore.org/api/idea/2210.json
- API Markdown: https://synthcore.org/api/idea/2210.md
- Verification status: unverified
- Source: [arXiv:2608.12839](https://arxiv.org/abs/2608.12839)
- Category: dynamics
- Solves: stability, generalization, accuracy
- ML areas: rnn, training-dynamics, regularization
- Math tags: stochastic-processes, probability, dynamical-systems, metric-geometry
- Ratings: usefulness 5/10; difficulty 4/10; novelty 4/10

## Idea description

Replace the usual fixed threshold or exponentially decaying adaptive threshold in a recurrent spiking layer with a signed reinforcement accumulator. Each spike updates a per-neuron state S by a signed increment, and the next spike requires membrane potential to overcome alpha times the positive part of S. This creates history-dependent negative feedback under sustained firing while retaining the ability of negative reinforcement to restore excitability.

## Mathematical statement

The paper state is Y_t=(V_t,S_t,K_t) in N^N x Z^N x N^N, where V_i(t) is membrane potential, S_i(t) is signed reinforcement memory, and K_i(t) is the number of reinforcement updates. On a firing event of neuron i, the extracted dynamics explicitly use S_i <- S_i + xi_i and K_i <- K_i + 1. The effective firing threshold is theta_i(t)=alpha S_i^+(t), with S_i^+=max{S_i,0}; alpha>=0 controls how strongly positive memory suppresses firing. The paper measures the population rate as r(t)=number of firings in (t-Delta,t] divided by N Delta and the mean threshold as theta_bar(t)=(alpha/N) sum_i S_i^+(t). For an ML adapter, use discrete time steps, membrane update V_i <- lambda V_i + I_i plus recurrent or synaptic input, emit z_i=1[V_i>=theta_i], then reset V_i <- (1-z_i)V_i and update S_i <- S_i + xi_i z_i. Choose xi_i=-b for a spike with probability p and xi_i=+a otherwise, or use the opposite sign convention after validating which event represents reinforcement; clip S to [-S_max,S_max] for numerical safety. The mathematical mechanism doing the work is state-dependent threshold feedback: positive S increases the threshold, whereas negative S cannot make the threshold negative.

## Key formulas

- $$Y_t=(V_t,S_t,K_t)\in\mathbb{N}^{N}\times\mathbb{Z}^{N}\times\mathbb{N}^{N}$$
- $$S_i\longmapsto S_i+\xi_i,\qquad K_i\longmapsto K_i+1$$
- $$\theta_i(t)=\alpha S_i^+(t),\qquad S_i^+=\max\{S_i,0\}$$
- $$r(t)=\frac{\text{number of firings in }(t-\Delta,t]}{N\Delta},\qquad \overline{\theta}(t)=\frac{\alpha}{N}\sum_{i=1}^{N}S_i^+(t)$$

## Implementation notes

(1) Integration point: modify the threshold-and-reset operation of a leaky integrate-and-fire or recurrent spiking layer. Maintain one float32 tensor S with shape [batch, neurons] in addition to membrane V; S is persistent across sequence steps and is reset only at episode boundaries, not at every truncated backpropagation window. At step t compute V_pre=lambda*V + synaptic_input, theta=alpha*relu(S), and spikes z=surrogate_heaviside(V_pre-theta). Reset V=(1-z)*V. (2) Pseudocode: `V = lam*V + I; theta = alpha*relu(S); z = spike_surrogate(V-theta); V = (1-z)*V; xi = where(bernoulli(p), -b, +a); S = clip(S + xi*z, -Smax, Smax); K = K + z`. Use a straight-through sigmoid or arctangent surrogate only for the derivative of z; detach xi and S updates if testing the pure dynamical effect, then compare with a fully differentiable S update. The baseline is an ALIF neuron with `S=beta*S+z` and a fixed learned threshold. (3) Computed from the paper: the positive-part threshold, signed accumulator, update-count interpretation, population-rate metric, and mean-threshold metric. Estimated empirically: firing-rate adaptation, firing-rate variance, gradient norm, and whether the recurrent state contracts; estimate contraction by measuring `||Y_t-Y'_t||_1` for two trajectories driven by the same inputs but different initial states. (4) First experiment: train a small 2-layer recurrent SNN on SHD or sequential MNIST with 256 hidden neurons, comparing LIF, ALIF, and Elephant adaptation at matched parameter count and surrogate-gradient settings. Sweep alpha in {0.1,0.5,1.0}, p in {0.6,0.75,0.9}, and b/a ratios. Evaluate validation accuracy, mean firing rate, firing-rate coefficient of variation, and exploding or vanishing gradient frequency. The desired signal is equal or better accuracy with lower sustained firing rate and lower recurrent-state variance; a negative result is increased threshold saturation, poor gradients, or no improvement over ALIF at equal spike count.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.