# Quenched-heterogeneity rectifying state-space layer

- ID: 3110
- Canonical URL: https://synthcore.org/idea/3110/quenched-heterogeneity-rectifying-state-space-layer
- API JSON: https://synthcore.org/api/idea/3110.json
- API Markdown: https://synthcore.org/api/idea/3110.md
- Verification status: unverified
- Source: [arXiv:2609.03560](https://arxiv.org/abs/2609.03560)
- Category: dynamics
- Solves: accuracy, sample-efficiency, stability
- ML areas: rnn, ssm, world-model, training-dynamics
- Math tags: statistical-mechanics, dynamical-systems, stochastic-processes, linear-algebra
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Build a recurrent or state-space layer with fixed, spatially heterogeneous relaxation rates and measure its steady response to positive versus negative input offsets. Unlike a homogeneous linear state-space model, an individual heterogeneous realization can generate a controlled even response, allowing directional transport, asymmetric temporal context, or hysteresis-like sequence transformations without inserting an explicit quadratic input feature.

## Mathematical statement

The paper uses quenched waiting times tau_i = tau_c exp(E_i/T), where E_i is a fixed trap depth, T is temperature, and tau_c is a microscopic time. Exponentially distributed trap depths induce psi_nu(tau) = nu tau_c^nu tau^(-1-nu) for tau >= tau_c, with nu = T/T_g > 1. For a fixed realization, expand the steady response around signed drive delta as J(delta; rho) = a1(rho) delta + a2(rho) delta^2 + a3(rho) delta^3 + ..., where rho is the baseline density or neural operating state. Spatial reflection maps position i to L+1-i. If the equilibrium profile obeys p_i = p_(L+1-i), then the paper predicts a2 = 0. The neural analogue uses h_(t+1) = M h_t + B u_t, with M = A + K, A_ii = exp(-Delta_t/tau_i), and K a local mixing matrix. The recurrence is contractive when the spectral radius rho(M) is less than one. Heterogeneity should be evaluated through the measured even-response coefficient a2, not assumed beneficial merely because weights are random.

## Key formulas

- $$\tau_i=\tau_c\exp(E_i/T),\qquad \phi(E)=T_g^{-1}\exp(-E/T_g),\qquad \psi_\nu(\tau)=\nu\tau_c^\nu\tau^{-1-\nu},\quad \tau\geq\tau_c,\quad \nu=T/T_g>1.$$
- $$J(\delta;\rho)=a_1(\rho)\delta+a_2(\rho)\delta^2+a_3(\rho)\delta^3+O(\delta^4),\qquad R(\delta)=\frac{J(\delta)+J(-\delta)}{2}=a_2(\rho)\delta^2+O(\delta^4).$$
- $$a_2=0\quad\text{if}\quad p_i=p_{L+1-i}\ \text{for every position }i.$$
- $$h_{t+1}=Mh_t+Bu_t,\qquad M=A+K,\qquad A_{ii}=\exp(-\Delta t/\tau_i),\qquad \rho(M)<1.$$

## Implementation notes

(1) Integration point: replace the diagonal transition matrix of a small recurrent or state-space model by a fixed quenched heterogeneous bank of time constants. For an input sequence u_t, use h_(t+1) = M h_t + B u_t and y_t = C h_t, initialize tau_i once, and keep these time constants fixed during training. Draw E_i from an exponential distribution with scale T_g, set tau_i = tau_c exp(E_i/T), and clip extreme values for numerical safety. Use a banded nearest-neighbor K so the hidden coordinates have a meaningful reflection operation. Rescale M after initialization to target spectral radius 0.8 or 0.9. (2) Pseudocode: sample E; compute tau; construct diagonal A; construct K; form M = A + K; estimate the largest eigenvalue magnitude; rescale K if needed; run the recurrence; train B, C, and optionally K by backpropagation. Add lambda times max(0, estimated_rho(M)-0.98)^2 if training pushes the system toward instability. For each held-out baseline state, evaluate converged outputs for drives +delta and -delta, fit a cubic polynomial, and record a2. (3) Computed from the paper: the Arrhenius time-scale construction and the symmetry prediction that a2 vanishes for a reflection-symmetric profile. Estimated empirically: spectral radius, equilibrium hidden profile p, profile asymmetry S = norm(p - R p), and response coefficients. Create a control by reflection-averaging two modules, one with tau_i and one with tau_(L+1-i). (4) First cheap experiment: compare 32-unit homogeneous, heterogeneous, and reflection-averaged models on a synthetic delayed directional-response task with target f(u) = u + 0.5 u^2, plus an adding problem and a sequence-classification task requiring asymmetric context. Match parameter counts and train with the same optimizer. Sweep delta from 0.01 to 0.2 and measure R(delta) = [J(delta)+J(-delta)]/2. The quantitative prediction is that R(delta)/delta^2 approaches a nonzero constant for an asymmetric realization, scales approximately linearly with S near S = 0, and falls to measurement noise in the reflection-averaged control. Independently sweep the spectral radius: hidden-state norms should remain bounded for rho(M) below one and grow rapidly after the predicted stability boundary rho(M) = 1.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.