Unconditionally Positive Gain-Loss Neural ODE

(1) Exact integration point: replace the vector-field evaluation and Euler or Runge-Kutta state update inside a latent Neural ODE, continuous-time VAE, recurrent state-space model, or differentiable simulator. The state tensor x must represent nonnegative quantities. Build two MLP heads, g_theta(x,t) and ell_phi(x,t), each with output dimension d. Apply softplus plus epsilon, for example 1e-6, to obtain the production tensor G and destruction tensor L.

(2) Pseudocode:

x = x0
for n in range(num_steps):
    G = softplus(g_theta(concat(x, t[n]))) + eps
    L = softplus(ell_phi(concat(x, t[n]))) + eps
    x = (x + h * G) / (1 + h * L)
    record(x)
loss = observation_loss(recorded_states, targets)
backpropagate(loss)

All operations in the update are elementwise. Initialize x0 with softplus or another nonnegative map. Do not clip x after the update, because clipping would conceal violations of the mathematical guarantee.

(3) Computed directly from the paper’s mathematics are the gain-loss decomposition, positivity-enforcing parameterization, and rational NSFD update. Estimated empirically are network capacity, epsilon, step size, and whether the learned rates reproduce target trajectories. Monitor the minimum coordinate of every trajectory, denominator magnitudes, gradient norms, one-step residuals, and the error after halving h. Once approximation error is controlled, first-order consistency predicts that discretization error should approximately halve when h is halved. Reverse-mode autodiff can train both rate networks because the update is explicit and differentiable.

(4) First cheap experiment: train on a latent SIR system with nonnegative states (S,I,R), using an MLP encoder and decoder. Compare an unconstrained NODE with fixed-step Euler or RK4 against the gain-loss NSFD model, matching parameter counts and function-evaluation budgets. Use 4, 8, and 16 steps over the same time interval, noisy partial observations, and several random seeds. Report validation trajectory MSE, the fraction of negative coordinates, final epidemic mass error, gradient failures, and wall-clock time. The expected signal is zero negative-state violations and substantially lower error or more stable training at coarse steps; a secondary success criterion is comparable fine-step accuracy with fewer function evaluations.