# Correlated stochastic integrate-and-fire recurrent layer

- ID: 648
- Canonical URL: https://synthcore.org/idea/648/correlated-stochastic-integrate-and-fire-recurrent-layer
- API JSON: https://synthcore.org/api/idea/648.json
- API Markdown: https://synthcore.org/api/idea/648.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.09575](https://arxiv.org/abs/2607.09575)
- Category: dynamics
- Solves: accuracy, stability, sample-efficiency
- ML areas: rnn, ssm, training-dynamics
- Math tags: stochastic-processes, probability, dynamical-systems, measure-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace a conventional leaky recurrent update with a population of stochastic membrane potentials that evolve only while subthreshold, emit an event at threshold, undergo a delayed reset, and receive feedback from a filtered population firing rate. Add a shared noise source alongside independent neuron noise to regularize the layer while preserving coordinated population-level dynamics.

## Mathematical statement

The core construction is the interacting particle system $$\mathrm{d}X_t^i=b(t,X_t^i,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}t+\sigma(t,X_t^i)\rho(t,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^0+\sigma(t,X_t^i)\sqrt{1-\rho^2(t,\nu_t^N,\mathfrak f_t^N)}\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^i-\mathrm{d}\sum_{k\ge1}\xi_k^i\mathbf1_{[0,t]}(\tau_k^i+\varsigma_k^i)$$. Here $X_t^i<0$ is the subthreshold membrane potential of neuron $i$, $b$ is its drift, $\sigma$ is the noise scale, $W^0$ is a shared Brownian motion, $W^i$ is independent neuron-specific Brownian motion, and $\rho\in[-1,1]$ controls shared-noise correlation. A spike occurs at $$\tau_k^i=\inf\{t>\tau_{k-1}^i+\varsigma_{k-1}^i:X_{t-}^i\ge0\},\qquad \tau_0^i=0,$$ with reset magnitude $\xi_k^i\ge0$ applied after refractory delay $\varsigma_k^i\ge0$. The population state and filtered firing signal are $$\nu_t^N=\frac1N\sum_{i=1}^N\delta_{X_t^i}\mathbf1_{\{X_t^i<0\}},\qquad \mathfrak f_t^N=\int_0^t\mathfrak K(t-s)\,\mathrm dF_s^N,$$ where $$F_t^N=\frac1N\sum_{i=1}^N J_t^i,\qquad J_t^i=\sum_{k\ge1}\mathbf1_{[0,t]}(\tau_k^i).$$ Thus $F_t^N$ is cumulative spike count per neuron, $\mathfrak K$ is a causal memory kernel, and $
u_t^N$ is the empirical subthreshold-potential measure. The adapted discrete equation uses Euler-Maruyama with shared Gaussian $z_t^0$ and independent Gaussians $z_t^i$.

## Key formulas

- $$\mathrm{d}X_t^i=b(t,X_t^i,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}t+\sigma(t,X_t^i)\rho(t,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^0+\sigma(t,X_t^i)\sqrt{1-\rho^2(t,\nu_t^N,\mathfrak f_t^N)}\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^i-\mathrm{d}\sum_{k\ge1}\xi_k^i\mathbf1_{[0,t]}(\tau_k^i+\varsigma_k^i)$$
- $$\tau_k^i=\inf\{t>\tau_{k-1}^i+\varsigma_{k-1}^i:X_{t-}^i\ge0\},\qquad \tau_0^i=0$$
- $$\nu_t^N=\frac1N\sum_{i=1}^N\delta_{X_t^i}\mathbf1_{\{X_t^i<0\}},\qquad \mathfrak f_t^N=\int_0^t\mathfrak K(t-s)\,\mathrm dF_s^N,\qquad F_t^N=\frac1N\sum_{i=1}^N J_t^i$$
- $$x_{t+\Delta t}^i=x_t^i+b_i\Delta t+\sigma_i\left(\rho_i z_t^0+\sqrt{1-\rho_i^2}\,z_t^i\right)\sqrt{\Delta t},\quad z_t^0,z_t^i\sim\mathcal N(0,1),\quad \text{if }x_t^i<0$$

## Implementation notes

Integrate this as a recurrent or state-space layer between input projections and the next network block. Maintain tensors `x[B,N]` for membrane states, `r[B,N]` for refractory counters, `s[B,N]` for spike indicators, and `f[B]` for the filtered population firing signal. At each time step, update the causal filter from the previous spike vector. For an exponential kernel $\mathfrak K(u)=\exp(-u/\tau_K)/\tau_K$, use `f = exp(-dt/tau_K)*f + mean(s)/tau_K`; for a sum of exponentials, maintain one such state per timescale. Construct the drift as `b_i = -x_i/tau_m + (W_in u_t)_i + (W_rec s_prev)_i + U_f f`, optionally adding a learned bounded function of the population statistics. For non-refractory neurons with `x<0`, sample one shared Gaussian `z0[B,1]` and independent Gaussians `zi[B,N]`, then apply the Euler-Maruyama formula with `rho=sigmoid(rho_logits)` or a fixed ablation value. Detect `spike = (x_new >= 0) & (r == 0)`, subtract reset magnitude `xi` or assign a fixed negative reset state, and set `r=round(varsiga/dt)`; while `r>0`, hold the membrane state fixed and decrement the counter. During training, use a fast-sigmoid or triangular surrogate derivative for the threshold, while inference uses binary events. The quantities computed directly from the paper are the shared/independent noise decomposition, threshold rule, refractory reset, empirical spike rate, and kernel convolution. The drift, noise scale, correlation, reset, and delay are learned or tuned parameters. First test on sequential MNIST or permuted sequential MNIST with a 128-neuron layer. Compare a deterministic leaky RNN, independent-noise spiking neurons with $\rho=0$, and the full shared-noise model at equal parameter count and recurrent steps. Track validation accuracy, gradient-norm variance, firing-rate variance, robustness to input noise, event sparsity, and wall-clock inference speed. A positive result is equal-or-better accuracy with lower gradient variance, improved noisy-input accuracy, or fewer active events at matched accuracy.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a correlated stochastic integrate-and-fire recurrent layer with shared/independent Euler-Maruyama noise, refractory reset, threshold spikes, and exponential population-rate filtering. The covariance identity and filter recurrence were numerically verified, with maximum errors 5.2e-6 and 5.6e-17 respectively. On the toy sequence task, correlated noise substantially reduced gradient-norm variance versus independent noise (0.0221 vs 1.1728) and firing activity (6.0% vs 9.0%), but did not match the deterministic baseline’s accuracy or improve noisy-input accuracy; the observed win is therefore limited to training-gradient variance and sparsity signals.

### Mechanism check

- Verdict: Built a correlated stochastic integrate-and-fire recurrent layer with shared/independent Euler-Maruyama noise, refractory reset, threshold spikes, and exponential population-rate filtering. The covariance identity and filter recurrence were numerically verified, with maximum errors 5.2e-6 and 5.6e-17 respectively. On the toy sequence task, correlated noise substantially reduced gradient-norm variance versus independent noise (0.0221 vs 1.1728) and firing activity (6.0% vs 9.0%), but did not match the deterministic baseline’s accuracy or improve noisy-input accuracy; the observed win is therefore limited to training-gradient variance and sparsity signals.
- Confidence: 7/10
- Limitations: Only one small synthetic sequence-classification task and one seed were tested; no sequential MNIST, parameter/FLOP-matched architecture audit, multi-seed confidence intervals, long-horizon stability study, or systematic rho/noise/refractory ablation was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_exp.py](https://synthcore.org/code/1427/bench_exp.py)
- [bench_report.json](https://synthcore.org/code/1427/bench_report.json)
- [experiment.py](https://synthcore.org/code/1427/experiment.py)
- [report.md](https://synthcore.org/code/1427/report.md)
- [report_bench_2026-09-04T210923.md](https://synthcore.org/code/1427/report_bench_2026-09-04T210923.md)
- [results.json](https://synthcore.org/code/1427/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1427)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.