# Permutation-family residual network

- ID: 946
- Canonical URL: https://synthcore.org/idea/946/permutation-family-residual-network
- API JSON: https://synthcore.org/api/idea/946.json
- API Markdown: https://synthcore.org/api/idea/946.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.15060](https://arxiv.org/abs/2607.15060)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: rnn, mlp, training-dynamics, data-augmentation
- Math tags: probability, statistics, linear-algebra, statistical-mechanics
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.

## Mathematical statement

Let p denote a configuration or permutation, c(p) its conjugacy-class or family label, s(p)=(-1)^{p_ex} its known sign, P_c the probability mass of family c, and E_c its conditional energy. The target signed observable is E_F=(sum_c s_c P_c E_c)/(sum_c s_c P_c), where s_c is constant within a family. Parameterize log P_c with a normalized model and decompose E_c=b_c(theta)+r_c(theta), where b_c is a hand-specified asymptotic physical baseline and r_c is learned by the neural network. Priors penalize deviations from known limiting behavior. The transferable mechanism is that cancellation occurs after low-dimensional family predictions are averaged, avoiding a ratio of two noisy Monte Carlo sample means.

## Key formulas

- $$\langle E\rangle_F=\frac{\langle \sigma E\rangle_B}{\langle\sigma\rangle_B}=\frac{\sum_c s_cP_cE_c}{\sum_c s_cP_c},\qquad s_c\in\{-1,+1\}.$$
- $$P_c(x)=\frac{\exp f_\phi(z_c(x))}{\sum_{c'}\exp f_\phi(z_{c'}(x))},\qquad E_c(x)=b_c(x;\theta)+r_\psi(z_c(x),h_{c-1}).$$
- $$\mathcal L=\sum_c\left[-n_c\log P_c+\frac{(\widehat E_c-E_c)^2}{2\widehat v_c}\right]+\lambda\sum_c\left\|b_c-b_c^{\mathrm{limit}}\right\|^2+\frac{\lambda_r}{2}\sum_c\|r_c\|^2.$$
- $$\operatorname{Var}[\widehat E_F]\ \text{for the direct ratio estimator becomes noise dominated when}\ \left|\sum_c s_cP_c\right|\lesssim O(n^{-1/2}),$$

## Implementation notes

1. Integration point: use this as a supervised or self-supervised head for examples possessing a known symmetry action, such as particle permutations, graph automorphisms, exchangeable sets, or structured data augmentations. Construct a family label from permutation cycle type or an orbit hash. A shared encoder produces family features; an LSTM is optional when families have an ordered cycle-length sequence. 2. Pseudocode: for each minibatch, compute family IDs c, signs s, counts n_c, and empirical conditional targets Ehat_c; compute logits f_phi, normalized probabilities P_c, baseline b_c, residual r_psi, and E_c=b_c+r_psi; minimize the displayed loss; estimate the final prediction as sum(s*P*E)/sum(s*P). Use numerical log-sum-exp stabilization and do not alter the target by clipping the denominator. 3. Computed from the paper’s mechanism: signed family aggregation, physical baseline plus residual decomposition, and asymptotic prior. Estimated empirically: family probabilities, conditional energies, residual uncertainty, and final aggregate variance. 4. First cheap experiment: use a permutation-invariant MLP on synthetic sets with known random-permutation signs. Compare direct signed regression, a vanilla LSTM, and the family-residual model while increasing cancelling negative families. The prediction is that the direct estimator fails near |D|=|sum_c s_cP_c| approximately n^{-1/2}, while the family model remains stable and its error grows with family-model error rather than 1/|D|. Test whether the measured crossover agrees with the predicted boundary within 20%.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a reusable PyTorch family-residual head in family_residual.py and a deterministic synthetic permutation-family experiment in experiment.py. The population family formula matched exactly (error 0), and the direct estimator became unstable near the predicted 1/sqrt(400)=0.05 boundary: RMSE rose sharply from 0.33 at D=0.20 to 8.42 at D=0.06. However, the direct log-log scaling was approximately D^-1.66 rather than the simple D^-1 expectation, and the family estimator also scaled approximately as 1/D because its conditional energy estimates remained noisy. Thus cancellation instability was reproduced, but the proposed architecture did not independently remove denominator sensitivity in this toy setting.

### Mechanism check

- Verdict: Built a reusable PyTorch family-residual head in family_residual.py and a deterministic synthetic permutation-family experiment in experiment.py. The population family formula matched exactly (error 0), and the direct estimator became unstable near the predicted 1/sqrt(400)=0.05 boundary: RMSE rose sharply from 0.33 at D=0.20 to 8.42 at D=0.06. However, the direct log-log scaling was approximately D^-1.66 rather than the simple D^-1 expectation, and the family estimator also scaled approximately as 1/D because its conditional energy estimates remained noisy. Thus cancellation instability was reproduced, but the proposed architecture did not independently remove denominator sensitivity in this toy setting.
- Confidence: 9/10
- Limitations: This is a small synthetic Monte Carlo verification, not a trained end-to-end permutation-invariant neural network or a real physical observable. Family probabilities were supplied exactly rather than learned, and no comparison of training FLOPs, generalization, or an LSTM/set encoder was performed. The experiment also did not test variance-aware training or a regime where the physical baseline is sufficiently accurate to suppress residual noise.

## Artifacts

- [experiment.py](https://synthcore.org/code/170/experiment.py)
- [family_residual.py](https://synthcore.org/code/170/family_residual.py)
- [report.md](https://synthcore.org/code/170/report.md)
- [results.json](https://synthcore.org/code/170/results.json)
- [Download all files as ZIP](https://synthcore.org/download/170)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.