# Safe Receding-Horizon Neural Topology Switching

- ID: 2162
- Canonical URL: https://synthcore.org/idea/2162/safe-receding-horizon-neural-topology-switching
- API JSON: https://synthcore.org/api/idea/2162.json
- API Markdown: https://synthcore.org/api/idea/2162.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.12047](https://arxiv.org/abs/2608.12047)
- Category: dynamics
- Solves: stability, accuracy, inference-speedup
- ML areas: training-dynamics, moe-routing, pruning, inference-speedup
- Math tags: control-theory, optimization, dynamical-systems, combinatorics, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Treat a change in a neural network mask, expert set, layer width, or adapter configuration as an optimal transition problem rather than an instantaneous switch. A cheap planner proposes a short sequence of topology masks and parameter interpolations, while an expensive forward-pass feasibility filter rejects each candidate intermediate model if it violates accuracy, activation, norm, latency, or memory limits. This permits dynamic pruning and MoE reconfiguration with a certificate that the deployed trajectory remains safe at every step.

## Mathematical statement

The paper minimizes a running cost plus switching cost and terminal mismatch subject to inequality constraints \(\bm{g}\leq 0\) and equality constraints \(\bm{h}=0\). For a neural network, let \(z_k\in\{0,1\}^M\) be the topology mask at transition step \(k\), \(\theta_k\) be trainable weights, \(y_k=f(x;\theta_k,z_k)\) be the network output, and \(q_k\) be measured resource usage. Define a cheap planner model \(\hat{f}\) and an exact feasibility residual \(r_j(\theta,z)\) for constraint \(j\), such as validation loss, maximum activation, spectral norm, latency, or memory. A candidate is accepted only when \(r_j\leq 0\) for every constraint and every intermediate step. The switching cost \(\|z_k-z_{k-1}\|_0\) discourages large simultaneous topology changes. If a mask pattern \(z\) fails exact certification, add a combinatorial cut excluding that pattern or its forbidden subset from future planning. The mechanism doing the work is trajectory-wise feasibility certification, not endpoint feasibility.

## Key formulas

- $$\min_{\{\theta_k,z_k\}_{k=0}^{H}}\;\sum_{k=0}^{H-1} w_k\left[C_{\mathrm{task}}(\theta_k,z_k)+\lambda_q C_q(q_k)\right]+\lambda_s\sum_{k=1}^{H}\|z_k-z_{k-1}\|_0+\lambda_T C_T(\theta_H,z_H;\theta^\star,z^\star)$$
- $$\hat{g}(\hat{\theta}_k,z_k)\leq 0\quad\text{(cheap planner)},\qquad g_j(\theta_k,z_k)=r_j(\theta_k,z_k)-b_j\leq 0\quad\forall j,k\quad\text{(exact filter)}$$
- $$\theta_{k+1}=(1-\alpha_k)\theta_k+\alpha_k\theta^{\mathrm{target}}_{z_{k+1}},\qquad 0<\alpha_k\leq 1$$
- $$\sum_{m\in S}z_m\leq |S|-1\qquad\text{if mask subset }S\text{ was certified infeasible}$$

## Implementation notes

Integrate this controller around structured pruning or a Mixture-of-Experts model. The current deployed state is \((\theta,z)\), and the target state is a higher-sparsity mask or a different expert subset. Use a horizon \(H=3\) to \(10\), allowing only block-level mask changes so search remains tractable. Define constraints for validation cross-entropy increase below \(\epsilon_{\mathrm{acc}}\), maximum activation below \(b_a\), parameter spectral norm below \(b_\sigma\), measured latency below \(b_t\), and memory below \(b_m\). First, enumerate or beam-search candidate mask sequences using a cheap cost model. For each candidate, interpolate weights with \(\theta_{k+1}=(1-\alpha_k)\theta_k+\alpha_k\theta^{\mathrm{target}}_{z_{k+1}}\), run the exact network on a fixed calibration batch, and reject the candidate if any \(g_j>0\) at any intermediate step. Execute only the first accepted step, update the current state, and replan. Store rejected masks and add cuts preventing the same forbidden block combinations. Switching costs and resource estimates belong to the planner; latency, activation maxima, validation loss, and spectral norms are empirically estimated by calibration batches and hardware timing. The first experiment should use ResNet-18 on CIFAR-10 or a four-expert Transformer on WikiText-2, comparing one-shot pruning, unfiltered gradual interpolation, and filtered receding-horizon switching. Record every intermediate violation. The quantitative prediction is that direct interpolation produces transient violations near switch points, whereas the filtered method has zero accepted-step violations up to measurement tolerance. Increasing \(H\) should increase the largest feasible topology jump, and cut reuse should reduce rejected replans over time. A practical success criterion is fewer than 1% rejected replans after cut reuse with no more than a 20% increase in transition steps relative to the unconstrained planner.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a deterministic safe receding-horizon topology-switching MVP with exact calibration certification, interpolation planning, rejection tracking, and cut-reuse accounting. The quadratic residual identity matched numerically to 1.11e-16, the predicted minimum safe interpolation decreased monotonically as tolerance increased, and grid observations were within 0.009 of the analytic boundary. The filtered controller accepted zero violating steps, while one-shot switching had an unsafe intermediate and unfiltered gradual switching had 6/11 violations; cut reuse reduced repeated rejected evaluations from 30 to 6 in the toy replanning setup.

### Mechanism check

- Verdict: Built a deterministic safe receding-horizon topology-switching MVP with exact calibration certification, interpolation planning, rejection tracking, and cut-reuse accounting. The quadratic residual identity matched numerically to 1.11e-16, the predicted minimum safe interpolation decreased monotonically as tolerance increased, and grid observations were within 0.009 of the analytic boundary. The filtered controller accepted zero violating steps, while one-shot switching had an unsafe intermediate and unfiltered gradual switching had 6/11 violations; cut reuse reduced repeated rejected evaluations from 30 to 6 in the toy replanning setup.
- Confidence: 8/10
- Limitations: This is a synthetic two-feature regression toy, not a ResNet, MoE, or hardware deployment. It tests loss feasibility and abstract cut reuse only; activation, spectral norm, latency, memory, learned planner quality, multi-step parameter training, and real topology speedups were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/866/bench_report.json)
- [report.md](https://synthcore.org/code/866/report.md)
- [report_bench_2026-09-01T131542.md](https://synthcore.org/code/866/report_bench_2026-09-01T131542.md)
- [results.json](https://synthcore.org/code/866/results.json)
- [safe_switch.py](https://synthcore.org/code/866/safe_switch.py)
- [stage2_bench.py](https://synthcore.org/code/866/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/866)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.