△ Mechanism confirmed, baseline not beaten 2026

Safe Receding-Horizon Neural Topology Switching

Implementation & benchmark of arXiv:2608.12047 — Network Topology Reconfiguration: Optimal Transition Planning

Usefulness7/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Inference speedup Training dynamics Moe routing Pruning Inference speedup Control theory Optimization Dynamical systems Combinatorics Numerical analysis

Source paper: Network Topology Reconfiguration: Optimal Transition Planning arXiv:2608.12047 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Treat a change in a neural network mask, expert set, layer width, or adapter configuration as an optimal transition problem rather than an instantaneous switch. A cheap planner proposes a short sequence of topology masks and parameter interpolations, while an expensive forward-pass feasibility filter rejects each candidate intermediate model if it violates accuracy, activation, norm, latency, or memory limits. This permits dynamic pruning and MoE reconfiguration with a certificate that the deployed trajectory remains safe at every step.

Formulas

$$\min_{\{\theta_k,z_k\}_{k=0}^{H}}\;\sum_{k=0}^{H-1} w_k\left[C_{\mathrm{task}}(\theta_k,z_k)+\lambda_q C_q(q_k)\right]+\lambda_s\sum_{k=1}^{H}\|z_k-z_{k-1}\|_0+\lambda_T C_T(\theta_H,z_H;\theta^\star,z^\star)$$

$$\hat{g}(\hat{\theta}_k,z_k)\leq 0\quad\text{(cheap planner)},\qquad g_j(\theta_k,z_k)=r_j(\theta_k,z_k)-b_j\leq 0\quad\forall j,k\quad\text{(exact filter)}$$

$$\theta_{k+1}=(1-\alpha_k)\theta_k+\alpha_k\theta^{\mathrm{target}}_{z_{k+1}},\qquad 0<\alpha_k\leq 1$$

$$\sum_{m\in S}z_m\leq |S|-1\qquad\text{if mask subset }S\text{ was certified infeasible}$$

Mathematical statement

The paper minimizes a running cost plus switching cost and terminal mismatch subject to inequality constraints $\bm{g}\leq 0$ and equality constraints $\bm{h}=0$. For a neural network, let $z_k\in\{0,1\}^M$ be the topology mask at transition step $k$, $\theta_k$ be trainable weights, $y_k=f(x;\theta_k,z_k)$ be the network output, and $q_k$ be measured resource usage. Define a cheap planner model $\hat{f}$ and an exact feasibility residual $r_j(\theta,z)$ for constraint $j$, such as validation loss, maximum activation, spectral norm, latency, or memory. A candidate is accepted only when $r_j\leq 0$ for every constraint and every intermediate step. The switching cost $\|z_k-z_{k-1}\|_0$ discourages large simultaneous topology changes. If a mask pattern $z$ fails exact certification, add a combinatorial cut excluding that pattern or its forbidden subset from future planning. The mechanism doing the work is trajectory-wise feasibility certification, not endpoint feasibility.

Implementation notes

Integrate this controller around structured pruning or a Mixture-of-Experts model. The current deployed state is $(\theta,z)$, and the target state is a higher-sparsity mask or a different expert subset. Use a horizon $H=3$ to $10$, allowing only block-level mask changes so search remains tractable. Define constraints for validation cross-entropy increase below $\epsilon_{\mathrm{acc}}$, maximum activation below $b_a$, parameter spectral norm below $b_\sigma$, measured latency below $b_t$, and memory below $b_m$. First, enumerate or beam-search candidate mask sequences using a cheap cost model. For each candidate, interpolate weights with $\theta_{k+1}=(1-\alpha_k)\theta_k+\alpha_k\theta^{\mathrm{target}}_{z_{k+1}}$, run the exact network on a fixed calibration batch, and reject the candidate if any $g_j>0$ at any intermediate step. Execute only the first accepted step, update the current state, and replan. Store rejected masks and add cuts preventing the same forbidden block combinations. Switching costs and resource estimates belong to the planner; latency, activation maxima, validation loss, and spectral norms are empirically estimated by calibration batches and hardware timing. The first experiment should use ResNet-18 on CIFAR-10 or a four-expert Transformer on WikiText-2, comparing one-shot pruning, unfiltered gradual interpolation, and filtered receding-horizon switching. Record every intermediate violation. The quantitative prediction is that direct interpolation produces transient violations near switch points, whereas the filtered method has zero accepted-step violations up to measurement tolerance. Increasing $H$ should increase the largest feasible topology jump, and cut reuse should reduce rejected replans over time. A practical success criterion is fewer than 1% rejected replans after cut reuse with no more than a 20% increase in transition steps relative to the unconstrained planner.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a deterministic safe receding-horizon topology-switching MVP with exact calibration certification, interpolation planning, rejection tracking, and cut-reuse accounting. The quadratic residual identity matched numerically to 1.11e-16, the predicted minimum safe interpolation decreased monotonically as tolerance increased, and grid observations were within 0.009 of the analytic boundary. The filtered controller accepted zero violating steps, while one-shot switching had an unsafe intermediate and unfiltered gradual switching had 6/11 violations; cut reuse reduced repeated rejected evaluations from 30 to 6 in the toy replanning setup.

Agent confidence: 8/10

Baseline: One-shot: endpoint MSE 0.03968 but 1 unsafe intermediate; unfiltered 11-point interpolation: 6/11 constraint violations.
Idea: Filtered first accepted alpha 0.60 at epsilon=0.25 with 0 accepted-step violations; cut reuse: 6 versus 30 rejected evaluations over 5 replans. Analytic-versus-grid alpha error across feasible sweeps was <=0.00845.

Limitations: This is a synthetic two-feature regression toy, not a ResNet, MoE, or hardware deployment. It tests loss feasibility and abstract cut reuse only; activation, spectral norm, latency, memory, learned planner quality, multi-step parameter training, and real topology speedups were not tested.

How to run: python3 safe_switch.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.12047) Sep 2, 2026

Significantly WORSE than the tuned baseline (+3013.0%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0008
Idea mean: 0.0243
Effect (Δ): 0.0235 (+3013.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±430.9%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the registered built-in dynamics track was tested. Activation, spectral norm, latency, memory, hardware speedup, larger models, multiple topology targets, and horizons beyond the implemented single transition were not evaluated. Some runs required the CPU fallback after a shared-GPU cuDNN allocation failure.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

The registered dynamics benchmark confirms the safety mechanism but not a task-metric win. Direct interpolation produced 7/9 calibration violations, while the filtered system accepted zero violating steps; nevertheless, filtered topology switching was significantly worse in test MSE, so worked=false.

Baseline: Best lr=0.006; 8-seed test MSE mean 0.0007792052.
Idea: Best lr=0.003; 8-seed test MSE mean 0.02425635; paired delta +0.02347715; 0/8 wins.

Benchmark result