Unverified 2026

Spectral-safe edge dropout

Implementation & benchmark of arXiv:2608.19145 — The critical probability for percolation on finite graphs

Usefulness5/10
Difficulty4/10
Novelty7/10

Source paper: The critical probability for percolation on finite graphs arXiv:2608.19145 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Calibrate random edge dropout in a GNN or sparse-attention layer using the spectral radius of the underlying communication graph. Retain edges with probability p chosen so that p lambda(A) is at least 1 plus a safety margin, preventing the random computation graph from entering a subcritical fragmented regime while retaining high sparsity.

Formulas

$$p\geqslant\frac{1+\varepsilon}{d(G)}\quad\Longrightarrow\quad \Pr\big[G_p\text{ has a component of order at least }\gamma d(G)\big]=1-o(1).$$
$$p_c(G)\approx\frac{1}{\lambda(G)},\qquad \lambda(G)=\lambda_{\max}(A),\qquad \rho\big(\mathbb{E}[A_p]\big)=\rho(pA)=p\lambda(G).$$
$$p_{\mathrm{safe}}=\min\left\{1,\frac{1+\varepsilon}{\widehat{\lambda}(A)}\right\},\qquad q_{t+1}=\frac{Aq_t}{\|Aq_t\|_2},\qquad \widehat{\lambda}(A)=\frac{\|Aq_T\|_2}{\|q_T\|_2}.$$

Mathematical statement

For a graph G = (V,E) with adjacency matrix A, lambda(G) = lambda_max(A) is the spectral radius, the largest eigenvalue of A. Under Bernoulli bond percolation, G_p is formed by independently retaining every edge with probability p. The paper's critical-scale principle is p_c(G) approximately equal to 1 divided by lambda(G): above p = (1 + epsilon) divided by lambda(G), the percolated graph is in a supercritical regime and typically contains a large component in the relevant graph regime, while below the threshold components are small. The extracted theorem also states that for every epsilon > 0 there exists gamma > 0 such that p >= (1 + epsilon) divided by d(G) implies that G_p has a component of order at least gamma d(G) with probability 1 - o(1), where d(G) = 2|E|/|V| is average degree. For a neural graph, use p_safe = min(1, (1 + epsilon) divided by lambda_hat(A)), where lambda_hat(A) is estimated by power iteration. The expected masked adjacency is E[A_p] = pA, whose spectral radius is p lambda(A).

Implementation notes

(1) Integration point: apply this rule to the binary edge mask immediately before message aggregation in a GNN, or immediately before constructing a sparse attention matrix over tokens, patches, or retrieved items. Let A be the fixed candidate adjacency or the binary graph induced by top-k attention connections. Do not modify the dense value projection; only mask communication edges. Always preserve diagonal self-loops. (2) Pseudocode: initialize a positive random vector q with unit norm; for t from 1 to T, compute z = A q and q = z divided by its Euclidean norm; after the final iteration compute lambda_hat = norm(A q); choose epsilon from {0.1, 0.25, 0.5}; set p = min(1, (1 + epsilon) divided by max(lambda_hat, 1e-6)); sample an independent Bernoulli mask M for non-diagonal candidate edges; set M_ii = 1; form A_p = A elementwise-multiplied by M; then use A_p in graph aggregation or add a large negative logit to dropped sparse-attention entries. (3) Computed from the paper's mathematics: the spectral-radius estimate, the threshold p lambda_hat(A) greater than 1, and the safety margin. Estimated empirically: the actual largest connected-component fraction, isolated-node rate, realized spectral radius, validation accuracy, and loss variance across masks. (4) First cheap experiment: train a 3-layer GCN on Cora or CiteSeer. Compare fixed dropout, average-degree calibration p = (1 + epsilon) divided by d(G), and spectral calibration p = (1 + epsilon) divided by lambda_hat(A), matching expected retained edges where possible. Sweep epsilon from 0 to 0.5 and report accuracy, loss variance, largest-component fraction, isolated-node fraction, and wall-clock time. The prediction is that spectral calibration reduces disconnected-mask failures and accuracy variance at high sparsity, especially on graphs whose degree distribution makes average degree a poor proxy for spectral radius.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.