✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Implementation & benchmark of arXiv:2607.01449 — Geometry-Aware R-Structured Kolmogorov-Arnold Networks

Usefulness6/10

Difficulty3/10

Novelty7/10

Architecture Accuracy Generalization Stability Mlp Regularization Training dynamics Geometry Optimization

Source paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Formulas

$$\hat y(\mathbf{x})=\alpha(\mathbf{x})h_{\mathrm{geom}}(\mathbf{x})+[1-\alpha(\mathbf{x})]h_{\mathrm{free}}(\mathbf{x})$$

$$\alpha(\mathbf{x})=\sigma(u_\phi(\mathbf{x})),\qquad \sigma(z)=\frac{1}{1+e^{-z}}$$

$$\mathcal{L}=\mathcal{L}_{\mathrm{task}}(\hat y,y)+\lambda\,\mathbb{E}_{\mathbf{x}}[\alpha(\mathbf{x})]$$

Mathematical statement

Let h_geom(x) be a predictor that receives or uses an R-function support score G(x), and let h_free(x) be an unconstrained KAN or MLP predictor. The agnostic architecture combines them with alpha(x) in [0,1]: yhat=alpha h_geom+(1-alpha)h_free. Parameterize alpha(x)=sigmoid(u_phi(x)), where u_phi is either a scalar beta for a global decision or a small mixer network for an input-dependent decision. A prior-suppression regularizer is L_gate=lambda E[alpha(x)], with lambda>=0; it penalizes unnecessary use of the geometric branch. The task loss remains L_task(yhat,y). The model therefore learns whether the geometric representation improves prediction rather than imposing it unconditionally.

Implementation notes

Create two parallel prediction heads sharing the input encoder. Compute h_free = free_net(x) and h_geom = geom_net(x, G(x)), where G(x) is built from the smoothed R-conjunction and R-disjunction formulas above. Add a global gate with alpha = sigmoid(beta), or an input-dependent gate with alpha = sigmoid(mixer_net(x)). Predict yhat = alpha*h_geom + (1-alpha)*h_free. Initialize beta=-4 so the model starts close to the unconstrained baseline and cannot be strongly harmed by a bad prior. Optimize the task loss jointly with the optional gate penalty lambda*mean(alpha). Use lambda=0 when the prior is trusted; otherwise sweep small positive values. Log mean alpha, gate entropy, the errors of both branches, and error conditioned on the true geometric region. The mathematical part is the convex interpolation and gate regularizer; choose gate capacity, beta initialization, lambda, and annealing empirically. The first cheap experiment should use four synthetic two-dimensional regression datasets: a correct disk prior, an incorrect shifted disk prior, a task with no geometric structure, and a task where only half the samples obey the prior. Compare against a free MLP or KAN, an always-geometric model, and a model that merely concatenates G(x) as a feature. Success means matching the geometry-aware model on correct-prior data while retaining baseline RMSE when the prior is wrong. The learned mean alpha should be high for reliable geometry and near zero when geometry is irrelevant.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a reproducible PyTorch MVP with free and geometry-aware branches, global or input-dependent sigmoid gates, and optional gate regularization. The math check confirmed convex interpolation, sigmoid(-4)=0.018 initialization, and the positive gradient of the suppression penalty. The input-dependent mixer improved over the free baseline on correct, wrong, irrelevant, and half-geometry tasks, while lowering mean alpha for wrong and irrelevant priors; the global gate was less effective.

Agent confidence: 7/10

Baseline: Free MLP RMSE: correct 0.1322, wrong 0.0673, none 0.0604, half 0.1670.
Idea: Input-dependent mixer RMSE: correct 0.0103 with alpha 0.608, wrong 0.0299 with alpha 0.180, none 0.0412 with alpha 0.106, half 0.1216 with alpha 0.718. Always-geometry RMSE: 0.0119, 0.0629, 0.0508, and 0.1225.

Limitations: Only one random seed and one small network/training schedule were tested. No repeated-seed intervals, FLOP or wall-clock comparison, noisy-prior sweep, larger datasets, or learned R-function support module were evaluated.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.01449) Sep 3, 2026

Beats the tuned baseline by 50.5% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 3.4863
Idea mean: 1.7247
Effect (Δ): -1.7616 (−50.5%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±15.1%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the structurally matched built-in tabular track was tested; vision, sequence, and dynamics tracks were not applicable. The idea has a larger parameter count than mlp_tiny, no parameter-matched control or wall-clock/FLOP normalization was run, and the fixed geometry prior was not swept over shifted or irrelevant variants in stage 2.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Worked ✓

Agent confidence: 8/10

Implemented the Agnostic Geometry-Prior Mixer and ran the full 8-seed tabular benchmark with a parity-matched learning-rate sweep. The idea achieved test MSE 1.7247 versus baseline 3.4863, with all 8 paired wins and permutation p=0.0081, satisfying the benchmark's significant-win criterion. The trained-model signature showed a nonconstant learned gate, though the mixer has more parameters and the branch errors indicate the gain should not be treated as isolated proof of the analytic prior itself.

Baseline: Friedman#1 tabular mlp_tiny, best lr=0.006, 30 epochs: mean test MSE 3.4863 ± 0.6332; per-seed [2.9687, 2.9661, 3.3915, 2.9757, 4.9135, 3.1062, 3.9271, 3.6418].
Idea: GeometryMixer, best lr=0.006, 30 epochs: mean test MSE 1.7247 ± 0.2112; per-seed [1.7298, 1.7004, 1.6195, 1.3495, 1.6963, 1.6350, 2.0825, 1.9848].
Mechanism signature: mean_alpha=0.2709, alpha_std=0.1770, alpha-support correlation=-0.0181, free-branch MSE=16.0480, geometry-branch MSE=71.3140, mixture test MSE=1.7298, confirmed=true
Paired delta: -1.7616 MSE; 8/8 wins; permutation p=0.0081.

Benchmark result