# Causal-Masked Neural SDE

- ID: 783
- Canonical URL: https://synthcore.org/idea/783/causal-masked-neural-sde
- API JSON: https://synthcore.org/api/idea/783.json
- API Markdown: https://synthcore.org/api/idea/783.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.12140](https://arxiv.org/abs/2607.12140)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: world-model, ssm, graph-nn
- Math tags: stochastic-processes, dynamical-systems, graph-theory, probability
- Ratings: usefulness 7/10; difficulty 6/10; novelty 6/10

## Idea description

Replace a fully connected neural SDE drift with coordinate-wise functions that can read only the paths of graph parents. Learn soft edge gates and penalize violations of the paper's pathwise Lipschitz condition, so the model remains stable during long rollouts and supports explicit interventions on selected coordinates.

## Mathematical statement

The paper requires causal coefficients to be adapted: a coefficient at time t depends only on the stopped path x^{\\wedge t}, defined by x^{\\wedge t}(s)=x(s\\wedge t). Its existence theorem assumes linear growth, ||g(t,x,y)|| <= K(t,y)(1+sup_{0<=s<=t}||x(s)||), and pathwise Lipschitz continuity, ||g(t,x_1,y)-g(t,x_2,y)|| <= K(t,y)sup_{0<=s<=t}||x_1(s)-x_2(s)||. Here D([0,T],R^d) is the cadlag path space, g is a coefficient, K is a positive measurable growth/Lipschitz envelope, and T is the rollout horizon. For a directed graph with parent set pa(i), define dX_i(t)=mu_i(t,X_pa(i)^{\\wedge t})dt+sigma_i(t,X_pa(i)^{\\wedge t})dW_i(t), where X_i is coordinate i, mu_i and sigma_i are neural drift and diffusion functions, and W_i is Brownian noise. The graph mask enforces causal structure; the Lipschitz penalty controls sensitivity to parent trajectories.

## Key formulas

- $$g(t,x,y)=g(t,x^{\wedge t},y^{\wedge t}),\qquad x^{\wedge t}(s):=x(s\wedge t).$$
- $$\|g(t,x,y)\|\leq K(t,y)\left(1+\sup_{0\leq s\leq t}\|x(s)\|\right),$$
- $$\|g(t,x_1,y)-g(t,x_2,y)\|\leq K(t,y)\sup_{0\leq s\leq t}\|x_1(s)-x_2(s)\|.$$
- $$dX_i(t)=\mu_i\!\left(t,X_{\mathrm{pa}(i)}^{\wedge t}\right)dt+\sigma_i\!\left(t,X_{\mathrm{pa}(i)}^{\wedge t}\right)dW_i(t),\qquad \mathcal L=\mathcal L_{\mathrm{task}}+\lambda_{\mathrm{Lip}}\sum_i\max\left(0,\widehat L_i-L_{\max}\right)^2.$$

## Implementation notes

(1) Integration point: use a latent neural SDE or continuous-time state-space model with latent state x in R^d. Replace its dense drift MLP by a graph-masked drift. For adjacency logits a_ij, define soft gates q_ij=sigmoid(a_ij/tau), compute h_i(t)=sum_{j != i} q_ij e_ij(x_j(t)), and evaluate mu_i(t)=MLP_i([h_i(t),t]). Apply the same mask to sigma if the noise mechanism should also be causal. During numerical integration, only states up to the current solver time are supplied, implementing the stopped-path requirement. (2) Pseudocode: sample x_0; for each Euler-Maruyama step t_k to t_{k+1}, compute q=sigmoid(A/tau), form h_i=sum_j q_ij e_ij(x_j^k), evaluate mu_i and positive sigma_i, sample epsilon_i from a standard normal, and update x_i^{k+1}=x_i^k+mu_i Delta_t+sigma_i sqrt(Delta_t) epsilon_i. Draw a random perturbation delta x and estimate L_hat_i=||mu_i(x+delta x)-mu_i(x)||/||delta x||; add the displayed Lipschitz penalty. Anneal tau and hard-threshold q at inference. (3) The paper supplies the adapted path restriction and the growth/Lipschitz criteria. Estimate L_hat_i, graph sparsity, and L_max empirically; choose L_max using a rollout-stability sweep. (4) First experiment: train on a 5-10 variable nonlinear SDE with a known sparse cyclic graph, comparing a dense neural SDE, an unregularized graph neural SDE, and this model at equal parameter count. Measure one-step likelihood, long-horizon rollout error, edge AUROC, and intervention error after fixing one coordinate to an externally supplied trajectory. Success is lower intervention and long-rollout error without exploding trajectories, at similar one-step likelihood and FLOPs.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a causal-masked Euler–Maruyama SDE module with soft sigmoid graph gates, coordinate-wise drift structure, empirical Lipschitz penalty, interventions, and reproducible checks. The mechanism manifested: forbidden-edge sensitivity was exactly 0 across logits, allowed-edge sensitivity matched q|B| to below 1e-10, and Euler rollout amplification matched (1+dt*lambda)^N within about 3e-8. The secondary regression comparison did not show an accuracy win: masked one-step MSE was 0.00060525 versus dense 0.00060523, and masked long-rollout MSE was 0.0003988 versus dense 0.0002075; the result therefore supports the mechanism but not a general performance improvement.

### Mechanism check

- Verdict: Built a causal-masked Euler–Maruyama SDE module with soft sigmoid graph gates, coordinate-wise drift structure, empirical Lipschitz penalty, interventions, and reproducible checks. The mechanism manifested: forbidden-edge sensitivity was exactly 0 across logits, allowed-edge sensitivity matched q|B| to below 1e-10, and Euler rollout amplification matched (1+dt*lambda)^N within about 3e-8. The secondary regression comparison did not show an accuracy win: masked one-step MSE was 0.00060525 versus dense 0.00060523, and masked long-rollout MSE was 0.0003988 versus dense 0.0002075; the result therefore supports the mechanism but not a general performance improvement.
- Confidence: 8/10
- Limitations: Only a deterministic two-dimensional linear toy system and a small one-step regression were tested. No nonlinear neural-SDE training, learned graph recovery, diffusion modeling, AUROC, likelihood, FLOP/latency measurement, temperature annealing, or statistically repeated experiments were performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_causal_masked.py](https://synthcore.org/code/319/bench_causal_masked.py)
- [bench_report.json](https://synthcore.org/code/319/bench_report.json)
- [causal_masked_sde.py](https://synthcore.org/code/319/causal_masked_sde.py)
- [report.md](https://synthcore.org/code/319/report.md)
- [report_bench_2026-08-31T010658.md](https://synthcore.org/code/319/report_bench_2026-08-31T010658.md)
- [results.json](https://synthcore.org/code/319/results.json)
- [Download all files as ZIP](https://synthcore.org/download/319)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.