Spectral-safe edge dropout
Implementation & benchmark of arXiv:2608.19145 — The critical probability for percolation on finite graphs
Source paper: The critical probability for percolation on finite graphs arXiv:2608.19145 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Calibrate random edge dropout in a GNN or sparse-attention layer using the spectral radius of the underlying communication graph. Retain edges with probability p chosen so that p lambda(A) is at least 1 plus a safety margin, preventing the random computation graph from entering a subcritical fragmented regime while retaining high sparsity.
Formulas
Mathematical statement
For a graph G = (V,E) with adjacency matrix A, lambda(G) = lambda_max(A) is the spectral radius, the largest eigenvalue of A. Under Bernoulli bond percolation, G_p is formed by independently retaining every edge with probability p. The paper's critical-scale principle is p_c(G) approximately equal to 1 divided by lambda(G): above p = (1 + epsilon) divided by lambda(G), the percolated graph is in a supercritical regime and typically contains a large component in the relevant graph regime, while below the threshold components are small. The extracted theorem also states that for every epsilon > 0 there exists gamma > 0 such that p >= (1 + epsilon) divided by d(G) implies that G_p has a component of order at least gamma d(G) with probability 1 - o(1), where d(G) = 2|E|/|V| is average degree. For a neural graph, use p_safe = min(1, (1 + epsilon) divided by lambda_hat(A)), where lambda_hat(A) is estimated by power iteration. The expected masked adjacency is E[A_p] = pA, whose spectral radius is p lambda(A).
Implementation notes
(1) Integration point: apply this rule to the binary edge mask immediately before message aggregation in a GNN, or immediately before constructing a sparse attention matrix over tokens, patches, or retrieved items. Let A be the fixed candidate adjacency or the binary graph induced by top-k attention connections. Do not modify the dense value projection; only mask communication edges. Always preserve diagonal self-loops. (2) Pseudocode: initialize a positive random vector q with unit norm; for t from 1 to T, compute z = A q and q = z divided by its Euclidean norm; after the final iteration compute lambda_hat = norm(A q); choose epsilon from {0.1, 0.25, 0.5}; set p = min(1, (1 + epsilon) divided by max(lambda_hat, 1e-6)); sample an independent Bernoulli mask M for non-diagonal candidate edges; set M_ii = 1; form A_p = A elementwise-multiplied by M; then use A_p in graph aggregation or add a large negative logit to dropped sparse-attention entries. (3) Computed from the paper's mathematics: the spectral-radius estimate, the threshold p lambda_hat(A) greater than 1, and the safety margin. Estimated empirically: the actual largest connected-component fraction, isolated-node rate, realized spectral radius, validation accuracy, and loss variance across masks. (4) First cheap experiment: train a 3-layer GCN on Cora or CiteSeer. Compare fixed dropout, average-degree calibration p = (1 + epsilon) divided by d(G), and spectral calibration p = (1 + epsilon) divided by lambda_hat(A), matching expected retained edges where possible. Sweep epsilon from 0 to 0.5 and report accuracy, loss variance, largest-component fraction, isolated-node fraction, and wall-clock time. The prediction is that spectral calibration reduces disconnected-mask failures and accuracy variance at high sparsity, especially on graphs whose degree distribution makes average degree a poor proxy for spectral radius.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.