Hard Sequential Neural ODE Solver

(1) Integration point: implement this as a replacement for the single trajectory network in a neural ODE, PINN, differentiable simulator, or diffusion/continuous-time world-model training loop. The state tensor is x in R^d, the control or conditioning tensor is a(t), and f is either a known vector field or the model's learned vector field. Use M local MLPs with shared architecture; do not detach b_m if training requires gradients through the full trajectory, but support a truncated-gradient option for memory control.

(2) Pseudocode:

partition tau[0:M] over [0,T]
b = x0
loss = 0
for m in 0..M-1:
    t = sample_collocation_points(tau[m], tau[m+1])
    z = MLP[m](t, condition)
    x = b + (t-tau[m]) * z
    dxdt = autodiff_time(x, t)
    r = dxdt - f(x, control(t), t)
    loss += omega[m] * mean(sum(r*r, dim=-1))
    b = b + (tau[m+1]-tau[m]) * MLP[m](tau[m+1], condition)
loss += task_loss(b, intermediate_x)
backprop(loss); optimizer.step()

For vector states, broadcast (t-tau[m]) across the state dimension. If enforcing a known terminal condition is also necessary, add a terminal residual or use a two-sided basis such as (t-tau[m])*(tau[m+1]-t) for a correction around an interpolant.

(3) Compute from the paper's mathematics: the local hard-anchor construction, ODE residual, and sequential propagation. Estimate empirically: choose M, segment lengths, residual weights omega[m], whether to stop gradients through b, and the number of collocation points. Start with uniform partitions, then compare an adaptive partition that splits the segment with largest residual or largest state increment.

(4) First cheap experiment: train a 4-layer MLP neural ODE on a long-horizon chaotic oscillator such as Lorenz-63 or a stiff Van der Pol system, comparing one global network, soft interface penalties, and hard sequential networks with M in {4, 8, 16}. Use identical total collocation points and Adam learning-rate schedules. Measure residual loss, trajectory error against a high-accuracy RK solver, gradient norm variance, and wall-clock time to a fixed error. Success is a lower failure rate and at least a 2x reduction in long-horizon trajectory error or substantially slower error growth at equal training FLOPs; also test whether exact continuity remains beneficial when M segments are trained jointly.