Screened Disordered Mixing Layer

Integrate this as a replacement for the dense input projection or token-mixing matrix in a small state-space model, recurrent sequence model, or graph neural network. Let the sequence length or reservoir width be N_p. Build a sparse graph, such as a 1D ring or 2D grid with nearest-neighbor and next-nearest-neighbor edges. Store positive trainable parameters C_i = softplus(c_i) + c_min and C_{ij} = softplus(e_{ij}) on graph edges. Construct the sparse symmetric matrix with diagonal C_i + sum_j C_{ij} and off-diagonal entries -C_{ij}. Use a learned sparse input matrix B in R^{N_p x d}, or map each input feature to a small set of nodes. At every forward pass, solve C Z = B with sparse Cholesky factorization or 10--30 conjugate-gradient iterations; then use Z x_t as the input drive. Normalize Z by estimating its largest singular value with two or three power iterations, giving K_tilde = Z/(||Z||_2 + epsilon). Update the state using the displayed alpha-interpolation rule, parameterizing alpha = sigmoid(a). Initialize c_i with two or three groups rather than identical values, then optionally add a small log-normal perturbation to create structural disorder. Train all parameters jointly, with a penalty lambda sum_{(i,j)} (log C_{ij} - mu)^2 to prevent the graph from disconnecting or becoming ill-conditioned. The paper supplies the capacitance construction and inverse-coupling mechanism; solver tolerance, normalization, memory coefficient, and regularization strength are engineering choices. First test on sequential MNIST or the Adding Problem using a 256-state dense linear-recurrent baseline and a 256-state local recurrent baseline. Compare equal parameter count and equal wall-clock budget. Success means lower validation loss at equal FLOPs, faster convergence, or matching accuracy with at least 2x fewer stored mixing parameters. Ablate homogeneous versus heterogeneous C_i, and measure response diversity using the effective rank of K and pairwise correlation of node activations.