# Partial-ReNoise Neural Architecture Mutation

- ID: 2761
- Canonical URL: https://synthcore.org/idea/2761/partial-renoise-neural-architecture-mutation
- API JSON: https://synthcore.org/api/idea/2761.json
- API Markdown: https://synthcore.org/api/idea/2761.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.27649](https://arxiv.org/abs/2608.27649)
- Category: architecture
- Solves: accuracy, scalability, sample-efficiency
- ML areas: cnn, transformer, mlp
- Math tags: stochastic-processes, probability, dynamical-systems
- Ratings: usefulness 8/10; difficulty 6/10; novelty 6/10

## Idea description

Replace independent architecture generation with a diffusion mutation kernel that starts from a known valid neural architecture, re-noises it for only a fraction of the diffusion horizon, and denoises it conditionally toward a new architecture. The resulting candidates should remain closer to the parent and retain validity at low mutation strength, while larger re-noising fractions should produce greater novelty and access to distinct architectural basins.

## Mathematical statement

Let x be a parent architecture encoding, x' a child architecture, T the total number of diffusion steps, and gamma in [0,1] the mutation fraction. Let q(x_gammaT | x) be the forward corruption distribution applied to the parent and p_theta(x_(t-1) | x_t,c) the learned reverse transition conditioned on task context c, such as parameter count or latency limits. The mutation kernel is K_gamma(x' | x) = integral q(x_gammaT | x) product from t=gammaT down to 1 of p_theta(x_(t-1) | x_t,c) dx_(1:gammaT). For a continuous embedding, use Gaussian forward corruption with alpha_bar_t equal to the product of (1-beta_s) from s=1 to t. For discrete graph tokens, use a categorical corruption kernel. The engineering prediction is that parent-child similarity decreases monotonically with gamma, while feasibility probability decreases much more slowly than novelty for small gamma.

## Key formulas

- $$K_{\gamma}(x'\mid x)=\int q(x_{\gamma T}\mid x)\prod_{t=\gamma T}^{1}p_{\theta}(x_{t-1}\mid x_t,c)\,dx_{1:\gamma T}.$$
- $$q(x_t\mid x)=\mathcal N\left(\sqrt{\bar\alpha_t}x,(1-\bar\alpha_t)I\right),\qquad \bar\alpha_t=\prod_{s=1}^{t}(1-\beta_s).$$
- $$x_{\gamma T}\leftarrow\sqrt{\bar\alpha_{\gamma T}}x+\sqrt{1-\bar\alpha_{\gamma T}}\,\epsilon,\quad \epsilon\sim\mathcal N(0,I),\qquad x_{t-1}\sim p_\theta(\cdot\mid x_t,c).$$
- $$\operatorname{accept}(x')=\mathbf 1\{\operatorname{Compile}(x')=1,\ \operatorname{Params}(x')\le B_p,\ \operatorname{Latency}(x')\le B_l,\ \operatorname{Acc}(x')\ge A_0+\Delta\}.$$

## Implementation notes

Use a dataset of valid CNN, MLP, or Transformer architectures encoded as fixed-length operation and edge tokens, or as a graph diffusion representation. Train a discrete diffusion model with conditioning vector c containing the target dataset, parameter budget, latency budget, and desired accuracy regime. During search, do not sample independently from the learned prior. Select an existing parent architecture x, choose gamma, corrupt it to timestep floor(gamma times T), and run the learned reverse model back to timestep zero. Pseudocode is: parent = archive.sample(); t = round(gamma*T); xt = forward_corrupt(parent,t); for s in range(t,0,-1): xt = reverse_sample(model,xt,s,context); child = decode(xt); if checker(child) and evaluate(child): archive.add(child). The checker must enforce acyclic graph structure, tensor-shape compatibility, parameter count, and hardware latency; measured validation accuracy is the final objective, not a model prediction. The paper contributes the anchored kernel and the separation between learned feasibility structure and external evaluation. Estimate the reverse model from architecture data and measure feasibility empirically with a held-out checking set. First experiment: use NAS-Bench-201 on CIFAR-10, train a small categorical diffusion model, and compare independent sampling against mutation from the best 20 architectures at gamma in {0.05, 0.15, 0.30, 0.60, 1.0}. Measure graph edit distance, validity rate, and benchmark accuracy. The predicted signature is monotonic growth of mean edit distance with gamma; for low gamma, validity should remain within 5 percentage points of the parent validity. As gamma approaches 1, behavior should approach independent-prior sampling, while gamma approaching 0 should produce near-duplicate children.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built a self-contained categorical partial re-noising architecture mutation MVP with validity checks, repair, an independent-prior baseline, and fixed-seed sweeps. The mechanism manifested: forward-token retention matched alpha_bar within 0.0028, edit distance increased monotonically from 0.000 at gamma=0 to 0.702 at gamma=1, and gamma=1 approached independent sampling with an edit-distance gap of 0.021. No accuracy improvement was demonstrated; post-repair validity was 100% by construction.

### Mechanism check

- Verdict: Built a self-contained categorical partial re-noising architecture mutation MVP with validity checks, repair, an independent-prior baseline, and fixed-seed sweeps. The mechanism manifested: forward-token retention matched alpha_bar within 0.0028, edit distance increased monotonically from 0.000 at gamma=0 to 0.702 at gamma=1, and gamma=1 approached independent sampling with an edit-distance gap of 0.021. No accuracy improvement was demonstrated; post-repair validity was 100% by construction.
- Confidence: 8/10
- Limitations: This is a toy fixed-length categorical architecture space, not NAS-Bench-201 or CIFAR-10. The reverse model is hand-designed rather than trained, latency and accuracy are synthetic proxies, only one random seed was used, and repair makes post-repair validity uninformative.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.9195
- Paired wins: 5/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_partial_renoise.py](https://synthcore.org/code/1060/bench_partial_renoise.py)
- [bench_report.json](https://synthcore.org/code/1060/bench_report.json)
- [partial_renoise_experiment.py](https://synthcore.org/code/1060/partial_renoise_experiment.py)
- [report.md](https://synthcore.org/code/1060/report.md)
- [report_bench_2026-09-01T234641.md](https://synthcore.org/code/1060/report_bench_2026-09-01T234641.md)
- [results.json](https://synthcore.org/code/1060/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1060)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.