Standard-Shadowing Regularizer for Neural ODEs

Integrate this at the latent rollout loss of a neural ODE, continuous-depth ResNet, or state-space model. Use a nominal initial state z0 and construct pseudo-trajectories by Euler or solver-step perturbations: z_tilde_{i+1}=z_tilde_i+Delta t_i f_theta(z_tilde_i)+xi_i, with bounded Gaussian or adversarial noise ||xi_i|| <= delta Delta t_i. Parameterize a monotone piecewise-linear h by positive slopes a_i=1+epsilon tanh(u_i), or directly project every slope into [1-epsilon,1+epsilon], and compare z_tilde_i to a dense interpolation of the nominal rollout at h(t_i). Add the hinge loss in the formula to the task loss; retain the slope penalty as a diagnostic rather than allowing arbitrary time warps. The paper supplies the exact secant-slope constraint and the distinction between oriented and standard shadowing. Estimate L_f, trajectory speed, and the actual tracking radius empirically from Jacobian-vector products and rollout errors. First test on a 2D neural ODE trained to model a damped oscillator, then on a small latent ODE for permuted-MNIST, comparing ordinary consistency training, oriented-shadowing consistency, and standard-shadowing regularization. Sweep epsilon and horizon T. The quantitative prediction is a degradation boundary: when epsilon falls below the empirical epsilon_crit approximately equal to rho divided by T L_f sup||f_theta||, standard-shadowing error should increase rapidly, while unrestricted monotone h should continue to give low oriented-shadowing error. Increasing T should move the boundary approximately as 1/T.