Hyperplane-Gated Piecewise Neural Dynamics
Implementation & benchmark of arXiv:2608.19785 — Learning piecewise-smooth dynamical systems
Source paper: Learning piecewise-smooth dynamical systems arXiv:2608.19785 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a single smooth neural vector field with a finite collection of smooth subnetworks selected by learned affine hyperplanes. The architecture exposes switching geometry directly, allowing it to represent friction-like or threshold dynamics without approximating discontinuities using excessively steep activations.
Formulas
Mathematical statement
Let the latent state be x in R^d and let R learned affine hyperplanes be H_r={x:n_r^T x=c_r}, with normal n_r in R^d and offset c_r in R. The hyperplane arrangement partitions state space into open convex cells P_k={x:A_k x<b_k}, where each row of A_k and corresponding entry of b_k encode the relevant hyperplane inequalities. Learn one smooth vector field f_k(x;theta_k) for each cell and define dx/dt=f_k(x;theta_k) for x in P_k. The transferable mechanism is explicit representation of the discontinuity set union_r H_r while each regional model remains smooth and can be regularized independently.
Implementation notes
Integrate this at the transition-function level of a neural ODE, continuous-time state-space model, or latent world model. Parameterize each switching function as s_r(x)=n_r^T x-c_r and each regional vector field as a small MLP f_k. For a fixed number R of hyperplanes, enumerate only the cells observed in the training data rather than constructing all O(R^d) possible cells. Compute signed distances s=Nx-c, assign a sign pattern, map that pattern to a regional expert, and return that expert's vector field. Train with one-step derivative loss plus multi-step rollout loss. During early training use a hysteresis band around each surface so routing does not oscillate from numerical noise; later anneal the band toward zero. Periodically refit hyperplanes using samples with high derivative residuals or abrupt changes, then jointly optimize hyperplanes and experts. The paper supplies the affine-arrangement construction and separation of geometry learning from smooth dynamics learning; N, c, cell assignments, and regional weights must be estimated from data. First cheap experiment: use a known 2D oscillator with a switch at x_1=0, comparing a smooth MLP, ReLU MLP, and this model. Measure derivative error versus signed distance to the true surface. The predicted signature is that the learned zero set should localize the switch within 10% of the state scale, while within-cell error decreases without requiring a very large network Lipschitz constant. A correct model should show a kink in error slope at the switching surface.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a hyperplane-gated two-expert dynamics model with learned threshold selection and compared it against smooth tanh and ReLU MLP baselines. Hysteresis-band coverage matched the predicted h scaling within 0.6%, boundary-offset MSE followed the predicted linear |delta| law with observed/predicted ratios 1.0025–1.0073, and regional affine recovery reached approximately 2e-29 MSE versus 0.0881 for one global affine model. On the matched toy derivative task, the gated model achieved approximately 4e-31 MSE, better than tanh MLP 0.00379 and ReLU MLP 0.00145; this demonstrates the intended representation benefit, though not broad stability or generalization.
- Agent confidence
- 9/10
- Baseline
- Smooth tanh MLP derivative MSE 0.0037908; ReLU MLP derivative MSE 0.0014469
- Idea
- Gated regional affine experts MSE 4.01e-31; learned boundary 0.0 versus true 0.0. Hysteresis coverage observed/predicted ratios 1.0012–1.006; boundary-offset MSE ratios 1.0025–1.0073; gated regional recovery MSE 1.96e-29 versus global affine 0.08806
Limitations: Only a synthetic 2-D single-hyperplane derivative-regression task was tested. The experiment did not test continuous-time rollout stability, multiple hyperplanes/cells, noisy observations, learned normals and offsets jointly, hysteresis during actual integration, FLOPs-normalized training, or out-of-distribution generalization.
How to run: python3 piecewise_dynamics_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.19785)
Significantly WORSE than the tuned baseline (+180.2%, p=0.0221, wins 1 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (+180.2%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0221 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±150.3%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the registered controlled-pendulum dynamics track was tested. The idea sweep used one routing temperature (8.0) and three learning rates; multiple hyperplanes/cells, hysteresis annealing, longer rollout horizons, and broader architectures were not tested.
How to run: python3 stage2_bench.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
Implemented and reran the matched dynamics benchmark on the registered built-in dynamics track. The hyperplane-gated GRU achieved mean test MSE 1.94363e-05 versus the tuned baseline GRU's 6.93655e-06; paired delta was +1.24998e-05 with permutation p=0.02205, yielding the canonical verdict idea worse (significant). The analytical boundary-offset check passed within 0.56%, but the trained-model mechanism signature was not confirmed and there was no benchmark win.
- Baseline
- rnn_small, best lr=0.01; sweep means: lr=0.001 -> 2.68613e-04, lr=0.003 -> 2.24300e-05, lr=0.01 -> 7.06617e-06; full 8-seed mean 6.93655e-06, std 1.29379e-06.
- Idea
- Hyperplane-gated GRU, best lr=0.01 and temperature=8.0; 8-seed mean 1.94363e-05, std 1.14324e-05; paired delta +1.24998e-05, p=0.02205, 1/8 wins.
Benchmark result
Significantly WORSE than the tuned baseline (+180.2%, p=0.0221, wins 1 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (+180.2%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0221 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±150.3%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the registered controlled-pendulum dynamics track was tested. The idea sweep used one routing temperature (8.0) and three learning rates; multiple hyperplanes/cells, hysteresis annealing, longer rollout horizons, and broader architectures were not tested.
How to run: python3 stage2_bench.py
Artifacts
- bench_report.json 4.9 KB View Raw
- piecewise_dynamics_experiment.py 6.0 KB View Raw
- report.md 1.7 KB View
- report_bench_2026-09-01T175225.md 4.9 KB View
- results.json 1.8 KB View Raw
- stage2_bench.py 6.6 KB View Raw
- stage2_rerun_stdout.json 4.9 KB View Raw