△ Mechanism confirmed, baseline not beaten 2026

Correlated stochastic integrate-and-fire recurrent layer

Implementation & benchmark of arXiv:2607.09575 — Probabilistic estimates for a system of noisy integrate-and-fire neurons

Usefulness6/10

Difficulty5/10

Novelty6/10

Dynamics Accuracy Stability Sample efficiency Rnn Ssm Training dynamics Stochastic processes Probability Dynamical systems Measure theory

Source paper: Probabilistic estimates for a system of noisy integrate-and-fire neurons arXiv:2607.09575 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a conventional leaky recurrent update with a population of stochastic membrane potentials that evolve only while subthreshold, emit an event at threshold, undergo a delayed reset, and receive feedback from a filtered population firing rate. Add a shared noise source alongside independent neuron noise to regularize the layer while preserving coordinated population-level dynamics.

Formulas

$$\mathrm{d}X_t^i=b(t,X_t^i,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}t+\sigma(t,X_t^i)\rho(t,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^0+\sigma(t,X_t^i)\sqrt{1-\rho^2(t,\nu_t^N,\mathfrak f_t^N)}\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^i-\mathrm{d}\sum_{k\ge1}\xi_k^i\mathbf1_{[0,t]}(\tau_k^i+\varsigma_k^i)$$

$$\tau_k^i=\inf\{t>\tau_{k-1}^i+\varsigma_{k-1}^i:X_{t-}^i\ge0\},\qquad \tau_0^i=0$$

$$\nu_t^N=\frac1N\sum_{i=1}^N\delta_{X_t^i}\mathbf1_{\{X_t^i<0\}},\qquad \mathfrak f_t^N=\int_0^t\mathfrak K(t-s)\,\mathrm dF_s^N,\qquad F_t^N=\frac1N\sum_{i=1}^N J_t^i$$

$$x_{t+\Delta t}^i=x_t^i+b_i\Delta t+\sigma_i\left(\rho_i z_t^0+\sqrt{1-\rho_i^2}\,z_t^i\right)\sqrt{\Delta t},\quad z_t^0,z_t^i\sim\mathcal N(0,1),\quad \text{if }x_t^i<0$$

Mathematical statement

The core construction is the interacting particle system $$\mathrm{d}X_t^i=b(t,X_t^i,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}t+\sigma(t,X_t^i)\rho(t,\nu_t^N,\mathfrak f_t^N)\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^0+\sigma(t,X_t^i)\sqrt{1-\rho^2(t,\nu_t^N,\mathfrak f_t^N)}\mathbf1_{\{X_t^i<0\}}\mathrm{d}W_t^i-\mathrm{d}\sum_{k\ge1}\xi_k^i\mathbf1_{[0,t]}(\tau_k^i+\varsigma_k^i)$$. Here $X_t^i<0$ is the subthreshold membrane potential of neuron $i$, $b$ is its drift, $\sigma$ is the noise scale, $W^0$ is a shared Brownian motion, $W^i$ is independent neuron-specific Brownian motion, and $\rho\in[-1,1]$ controls shared-noise correlation. A spike occurs at $$\tau_k^i=\inf\{t>\tau_{k-1}^i+\varsigma_{k-1}^i:X_{t-}^i\ge0\},\qquad \tau_0^i=0,$$ with reset magnitude $\xi_k^i\ge0$ applied after refractory delay $\varsigma_k^i\ge0$. The population state and filtered firing signal are $$\nu_t^N=\frac1N\sum_{i=1}^N\delta_{X_t^i}\mathbf1_{\{X_t^i<0\}},\qquad \mathfrak f_t^N=\int_0^t\mathfrak K(t-s)\,\mathrm dF_s^N,$$ where $$F_t^N=\frac1N\sum_{i=1}^N J_t^i,\qquad J_t^i=\sum_{k\ge1}\mathbf1_{[0,t]}(\tau_k^i).$$ Thus $F_t^N$ is cumulative spike count per neuron, $\mathfrak K$ is a causal memory kernel, and $
u_t^N$ is the empirical subthreshold-potential measure. The adapted discrete equation uses Euler-Maruyama with shared Gaussian $z_t^0$ and independent Gaussians $z_t^i$.

Implementation notes

Integrate this as a recurrent or state-space layer between input projections and the next network block. Maintain tensors x[B,N] for membrane states, r[B,N] for refractory counters, s[B,N] for spike indicators, and f[B] for the filtered population firing signal. At each time step, update the causal filter from the previous spike vector. For an exponential kernel $\mathfrak K(u)=\exp(-u/\tau_K)/\tau_K$, use f = exp(-dt/tau_K)*f + mean(s)/tau_K; for a sum of exponentials, maintain one such state per timescale. Construct the drift as b_i = -x_i/tau_m + (W_in u_t)_i + (W_rec s_prev)_i + U_f f, optionally adding a learned bounded function of the population statistics. For non-refractory neurons with x<0, sample one shared Gaussian z0[B,1] and independent Gaussians zi[B,N], then apply the Euler-Maruyama formula with rho=sigmoid(rho_logits) or a fixed ablation value. Detect spike = (x_new >= 0) & (r == 0), subtract reset magnitude xi or assign a fixed negative reset state, and set r=round(varsiga/dt); while r>0, hold the membrane state fixed and decrement the counter. During training, use a fast-sigmoid or triangular surrogate derivative for the threshold, while inference uses binary events. The quantities computed directly from the paper are the shared/independent noise decomposition, threshold rule, refractory reset, empirical spike rate, and kernel convolution. The drift, noise scale, correlation, reset, and delay are learned or tuned parameters. First test on sequential MNIST or permuted sequential MNIST with a 128-neuron layer. Compare a deterministic leaky RNN, independent-noise spiking neurons with $\rho=0$, and the full shared-noise model at equal parameter count and recurrent steps. Track validation accuracy, gradient-norm variance, firing-rate variance, robustness to input noise, event sparsity, and wall-clock inference speed. A positive result is equal-or-better accuracy with lower gradient variance, improved noisy-input accuracy, or fewer active events at matched accuracy.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a correlated stochastic integrate-and-fire recurrent layer with shared/independent Euler-Maruyama noise, refractory reset, threshold spikes, and exponential population-rate filtering. The covariance identity and filter recurrence were numerically verified, with maximum errors 5.2e-6 and 5.6e-17 respectively. On the toy sequence task, correlated noise substantially reduced gradient-norm variance versus independent noise (0.0221 vs 1.1728) and firing activity (6.0% vs 9.0%), but did not match the deterministic baseline’s accuracy or improve noisy-input accuracy; the observed win is therefore limited to training-gradient variance and sparsity signals.

Agent confidence: 7/10

Baseline: Deterministic leaky RNN: accuracy 0.9551, noisy accuracy 0.5000, gradient-norm variance 0.0471
Idea: Correlated IF (rho initialized 0.65): accuracy 0.7246, noisy accuracy 0.5137, gradient-norm variance 0.0221, firing rate/activity 0.0600

Limitations: Only one small synthetic sequence-classification task and one seed were tested; no sequential MNIST, parameter/FLOP-matched architecture audit, multi-seed confidence intervals, long-horizon stability study, or systematic rho/noise/refractory ablation was performed.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.09575) Sep 4, 2026

Significantly WORSE than the tuned baseline (+3668.3%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0019
Idea mean: 0.0703
Effect (Δ): 0.0684 (+3668.3%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±1389.0%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested, with 400 training and 200 test examples, 8 epochs, hidden width 64, and three learning rates. No longer-horizon rollout, refractory/noise/rho ablation, larger dataset, or additional task track was tested.

How to run: python3 bench_exp.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented the correlated stochastic integrate-and-fire recurrent layer and evaluated it against a matched deterministic leaky recurrent system on the structurally appropriate dynamics track. The trained-model mechanism signature was confirmed quantitatively: observed shared-noise covariance factor 0.42043 versus predicted 0.42250. The idea nevertheless lost significantly on dynamics MSE, 0.0702823 versus 0.00186508, paired delta +0.0684172 with permutation p=0.0081, so it does not provide a benchmark win.

Baseline: dynamics MSE mean 0.00186508 over 8 paired seeds; best lr=0.01
Idea: dynamics MSE mean 0.07028228 over 8 paired seeds; best lr=0.01; mean spike rate 0.03861
Mechanism signature: predicted covariance factor 0.42250; observed trained-model mean 0.42043; confirmed=true

Benchmark result