Data-Identified Neural Dependency Graph
Implementation & benchmark of arXiv:2608.16759 — Novel methodology for obtaining design structure matrices using network identification
Source paper: Novel methodology for obtaining design structure matrices using network identification arXiv:2608.16759 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Partition a neural state or feature vector into blocks and identify directed dependencies between blocks from one-step transition data. Use the inferred design structure matrix as a hard mask or soft gate on recurrent, state-space, graph, or mixture-of-experts couplings, replacing a dense unconstrained interaction matrix with a data-supported sparse graph.
Formulas
Mathematical statement
A design structure matrix is represented by an adjacency matrix Adj, where Adj[i,j] = 1 denotes a directed dependency from source node j to target node i, and Adj[i,j] = 0 denotes no dependency. For a neural state z_t partitioned into K blocks, fit the linear identification surrogate z_(t+1) = A z_t + B u_t + v_t, where A[i,j] is the estimated influence of source block j on target block i, u_t is an external input, B is its coupling matrix, and v_t is process noise. Construct a binary mask D[i,j] = 1 when the estimated coupling magnitude exceeds a statistically chosen threshold. The masked neural transition uses only D[i,j] = 1 block interactions. The useful mechanism is sparse support identification: it reduces interaction count from K^2 to the number of retained edges while testing whether removed dependencies increase prediction or rollout error.
Implementation notes
1. Integration point: use a recurrent neural network, SSM, graph neural network, or MoE with K hidden blocks of equal width. First train or initialize a small dense model and collect trajectories containing z_t, z_(t+1), and u_t. Identify dependencies at block level rather than scalar-weight level because block-level coefficients are more statistically reliable. 2. Pseudocode: fit a ridge or LASSO regression z_(t+1) approximately equals A z_t + B u_t on the collected trajectories; bootstrap trajectory subsets or use held-out folds; estimate each coupling A_hat[i,j] and standard error s[i,j]; set D[i,j] = 1 when abs(A_hat[i,j]) > c times s[i,j], or retain the top q edges per target block; replace every dense block W[i,j] by D[i,j] times W[i,j]; fine-tune surviving weights using the task loss and a small transition-prediction loss. 3. Computed from the paper mechanism: the directed adjacency or DSM and conversion between weighted and binary dependencies. Estimated empirically: A, uncertainty thresholds, sparsity, and nonlinear fine-tuning effects. 4. First experiment: train an 8-block GRU or linear SSM on Lorenz-63, a five-node coupled-oscillator system, or a synthetic sparse linear dynamical system with known graph. Compare dense, random-mask, magnitude-pruned, and identified-mask models at equal parameter budgets. Sweep c or tau, measure graph precision and recall, one-step error, 50-step rollout error, and wall-clock time. Prediction: false-positive edges should increase sharply when the threshold drops below the coefficient noise floor; at the bootstrap threshold, at least 80 percent of true block edges should be recovered, while retaining 20 to 40 percent of dense edges should keep one-step error within 10 percent and reduce interaction cost approximately by the retained edge fraction.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.