# Agnostic Geometry-Prior Mixer

- ID: 333
- Canonical URL: https://synthcore.org/idea/333/agnostic-geometry-prior-mixer
- API JSON: https://synthcore.org/api/idea/333.json
- API Markdown: https://synthcore.org/api/idea/333.md
- Verification status: beats_baseline
- Source: [arXiv:2607.01449](https://arxiv.org/abs/2607.01449)
- Category: architecture
- Solves: accuracy, generalization, stability
- ML areas: mlp, regularization, training-dynamics
- Math tags: geometry, optimization
- Ratings: usefulness 6/10; difficulty 3/10; novelty 7/10

## Idea description

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

## Mathematical statement

Let h_geom(x) be a predictor that receives or uses an R-function support score G(x), and let h_free(x) be an unconstrained KAN or MLP predictor. The agnostic architecture combines them with alpha(x) in [0,1]: yhat=alpha h_geom+(1-alpha)h_free. Parameterize alpha(x)=sigmoid(u_phi(x)), where u_phi is either a scalar beta for a global decision or a small mixer network for an input-dependent decision. A prior-suppression regularizer is L_gate=lambda E[alpha(x)], with lambda>=0; it penalizes unnecessary use of the geometric branch. The task loss remains L_task(yhat,y). The model therefore learns whether the geometric representation improves prediction rather than imposing it unconditionally.

## Key formulas

- $$\hat y(\mathbf{x})=\alpha(\mathbf{x})h_{\mathrm{geom}}(\mathbf{x})+[1-\alpha(\mathbf{x})]h_{\mathrm{free}}(\mathbf{x})$$
- $$\alpha(\mathbf{x})=\sigma(u_\phi(\mathbf{x})),\qquad \sigma(z)=\frac{1}{1+e^{-z}}$$
- $$\mathcal{L}=\mathcal{L}_{\mathrm{task}}(\hat y,y)+\lambda\,\mathbb{E}_{\mathbf{x}}[\alpha(\mathbf{x})]$$

## Implementation notes

Create two parallel prediction heads sharing the input encoder. Compute `h_free = free_net(x)` and `h_geom = geom_net(x, G(x))`, where G(x) is built from the smoothed R-conjunction and R-disjunction formulas above. Add a global gate with `alpha = sigmoid(beta)`, or an input-dependent gate with `alpha = sigmoid(mixer_net(x))`. Predict `yhat = alpha*h_geom + (1-alpha)*h_free`. Initialize beta=-4 so the model starts close to the unconstrained baseline and cannot be strongly harmed by a bad prior. Optimize the task loss jointly with the optional gate penalty `lambda*mean(alpha)`. Use lambda=0 when the prior is trusted; otherwise sweep small positive values. Log mean alpha, gate entropy, the errors of both branches, and error conditioned on the true geometric region. The mathematical part is the convex interpolation and gate regularizer; choose gate capacity, beta initialization, lambda, and annealing empirically. The first cheap experiment should use four synthetic two-dimensional regression datasets: a correct disk prior, an incorrect shifted disk prior, a task with no geometric structure, and a task where only half the samples obey the prior. Compare against a free MLP or KAN, an always-geometric model, and a model that merely concatenates G(x) as a feature. Success means matching the geometry-aware model on correct-prior data while retaining baseline RMSE when the prior is wrong. The learned mean alpha should be high for reliable geometry and near zero when geometry is irrelevant.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a reproducible PyTorch MVP with free and geometry-aware branches, global or input-dependent sigmoid gates, and optional gate regularization. The math check confirmed convex interpolation, sigmoid(-4)=0.018 initialization, and the positive gradient of the suppression penalty. The input-dependent mixer improved over the free baseline on correct, wrong, irrelevant, and half-geometry tasks, while lowering mean alpha for wrong and irrelevant priors; the global gate was less effective.

### Mechanism check

- Verdict: Built a reproducible PyTorch MVP with free and geometry-aware branches, global or input-dependent sigmoid gates, and optional gate regularization. The math check confirmed convex interpolation, sigmoid(-4)=0.018 initialization, and the positive gradient of the suppression penalty. The input-dependent mixer improved over the free baseline on correct, wrong, irrelevant, and half-geometry tasks, while lowering mean alpha for wrong and irrelevant priors; the global gate was less effective.
- Confidence: 7/10
- Limitations: Only one random seed and one small network/training schedule were tested. No repeated-seed intervals, FLOP or wall-clock comparison, noisy-prior sweep, larger datasets, or learned R-function support module were evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1267/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1267/bench_report.json)
- [experiment.py](https://synthcore.org/code/1267/experiment.py)
- [report.md](https://synthcore.org/code/1267/report.md)
- [report_bench_2026-09-03T235914.md](https://synthcore.org/code/1267/report_bench_2026-09-03T235914.md)
- [results.json](https://synthcore.org/code/1267/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1267)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.