Convex-order stochastic expert layer

(1) Integration point: insert the module as a stochastic residual block in a small MoE transformer, graph encoder, or MLP-MoE. Maintain K expert states h[u] in R^d for each token or batch item. Let W be a K×K nonnegative row-normalized interaction matrix and q a learned projection. (2) Pseudocode: initialize h; for m in 1..M, compute c[u]=Σ_v W[u,v]q(h[v]); compute drift=difference network b(h[u],c[u]); compute diagonal diffusion s[u]=softplus(g(h[u],c[u]))+δ; sample ξ[u]~N(0,I); set h[u]=h[u]+dt*b+sqrt(dt)*s[u]*ξ[u]. Produce candidate samples y and reference samples x using shared noise where possible. For R convex probes φ_r, calculate C[r]=mean(φ_r(y))−mean(φ_r(x)); add α times mean(ReLU(ε−C[r])) plus λ times squared mean difference to the task loss. Use φ_r(a)=|a_j|, a_j², ReLU(w_r^T a+t_r), and (w_r^T a)². (3) The exact paper machinery is the Euler update and convex-order definition; the all-convex condition is estimated empirically through R probes. Estimate W with a low-rank parameterization W=rownorm(softplus(AB^T)) to keep cost manageable. (4) First experiment: K=8, CIFAR-10, four-layer MLP-MoE, comparing deterministic MoE, independent Gaussian residual noise, and this layer at matched parameter count. Record accuracy, expected calibration error, robustness under input noise, mean drift, convex-probe violations, and loss variance across seeds. Success means better calibration or corruption accuracy at equal clean accuracy, fewer violations across depth, and less than 10% training overhead.