Quenched-heterogeneity rectifying state-space layer
Implementation & benchmark of arXiv:2609.03560 — Sample-specific rectification-like response in a boundary-driven exclusion process
Source paper: Sample-specific rectification-like response in a boundary-driven exclusion process arXiv:2609.03560 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Build a recurrent or state-space layer with fixed, spatially heterogeneous relaxation rates and measure its steady response to positive versus negative input offsets. Unlike a homogeneous linear state-space model, an individual heterogeneous realization can generate a controlled even response, allowing directional transport, asymmetric temporal context, or hysteresis-like sequence transformations without inserting an explicit quadratic input feature.
Formulas
Mathematical statement
The paper uses quenched waiting times tau_i = tau_c exp(E_i/T), where E_i is a fixed trap depth, T is temperature, and tau_c is a microscopic time. Exponentially distributed trap depths induce psi_nu(tau) = nu tau_c^nu tau^(-1-nu) for tau >= tau_c, with nu = T/T_g > 1. For a fixed realization, expand the steady response around signed drive delta as J(delta; rho) = a1(rho) delta + a2(rho) delta^2 + a3(rho) delta^3 + ..., where rho is the baseline density or neural operating state. Spatial reflection maps position i to L+1-i. If the equilibrium profile obeys p_i = p_(L+1-i), then the paper predicts a2 = 0. The neural analogue uses h_(t+1) = M h_t + B u_t, with M = A + K, A_ii = exp(-Delta_t/tau_i), and K a local mixing matrix. The recurrence is contractive when the spectral radius rho(M) is less than one. Heterogeneity should be evaluated through the measured even-response coefficient a2, not assumed beneficial merely because weights are random.
Implementation notes
(1) Integration point: replace the diagonal transition matrix of a small recurrent or state-space model by a fixed quenched heterogeneous bank of time constants. For an input sequence u_t, use h_(t+1) = M h_t + B u_t and y_t = C h_t, initialize tau_i once, and keep these time constants fixed during training. Draw E_i from an exponential distribution with scale T_g, set tau_i = tau_c exp(E_i/T), and clip extreme values for numerical safety. Use a banded nearest-neighbor K so the hidden coordinates have a meaningful reflection operation. Rescale M after initialization to target spectral radius 0.8 or 0.9. (2) Pseudocode: sample E; compute tau; construct diagonal A; construct K; form M = A + K; estimate the largest eigenvalue magnitude; rescale K if needed; run the recurrence; train B, C, and optionally K by backpropagation. Add lambda times max(0, estimated_rho(M)-0.98)^2 if training pushes the system toward instability. For each held-out baseline state, evaluate converged outputs for drives +delta and -delta, fit a cubic polynomial, and record a2. (3) Computed from the paper: the Arrhenius time-scale construction and the symmetry prediction that a2 vanishes for a reflection-symmetric profile. Estimated empirically: spectral radius, equilibrium hidden profile p, profile asymmetry S = norm(p - R p), and response coefficients. Create a control by reflection-averaging two modules, one with tau_i and one with tau_(L+1-i). (4) First cheap experiment: compare 32-unit homogeneous, heterogeneous, and reflection-averaged models on a synthetic delayed directional-response task with target f(u) = u + 0.5 u^2, plus an adding problem and a sequence-classification task requiring asymmetric context. Match parameter counts and train with the same optimizer. Sweep delta from 0.01 to 0.2 and measure R(delta) = [J(delta)+J(-delta)]/2. The quantitative prediction is that R(delta)/delta^2 approaches a nonzero constant for an asymmetric realization, scales approximately linearly with S near S = 0, and falls to measurement noise in the reflection-averaged control. Independently sweep the spectral radius: hidden-state norms should remain bounded for rho(M) below one and grow rapidly after the predicted stability boundary rho(M) = 1.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.