Linear-solve ensemble controller
Implementation & benchmark of arXiv:2607.21112 — Exact ensemble controllability for neural differential equations via neural interpolation
Source paper: Exact ensemble controllability for neural differential equations via neural interpolation arXiv:2607.21112 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add a shallow neural interpolation controller to a neural ODE or state-space model so one shared vector field matches prescribed derivatives at several anchor trajectories. At every control time, compute controller weights from a small linear system instead of learning all task-specific parameters by backpropagation.
Formulas
Mathematical statement
The paper studies neural dynamics y'(t) = L y(t) + F(t,y), where y(t) is the state, L is a linear operator, and F is a shallow neural-network source term. Its construction uses an interaction function G_0(a,b) induced by the sigmoid network and an interpolation matrix P(t) whose entries are P_ij(t) = G_0(y_des_i(t), y_des_j(t)) with diagonal entries P_ii(t) = 1. For two desired trajectories, the paper obtains w^1(t) = (1/(1 - G_0(y_des_1(t), y_des_2(t)), 0)^T and w^2(t) = (-1/(1 - G_0(y_des_1(t), y_des_2(t)), 0)^T, giving F_1(t,y) = [G_0(y,y_des_1(t)) - G_0(y,y_des_2(t))] / [1 - G_0(y_des_1(t), y_des_2(t))] and F_2(t,y) = 0. The transferable property is exact finite-point interpolation whenever P(t) is nonsingular; the denominator shows that nearly dependent anchor states create unstable coefficients. In the ML adaptation, z_i(t) in R^d are anchor states, v_i(t) in R^d are desired vector-field values, K_ij(t) = G_0(z_i(t), z_j(t)), V is the matrix whose i-th row is v_i(t)^T, and A(t) solves K(t) A(t) = V.
Implementation notes
Integrate this at the residual-vector-field operation of a small neural ODE, latent SSM, or continuous-depth residual block. Let the base dynamics be f_theta(t,y) and add F_interp(t,y). Choose N anchor examples or task-support states z_i(t) and desired vector-field values v_i(t). These values can come from a teacher trajectory, a target latent trajectory, or finite-difference derivatives of labeled target states. At each ODE solver evaluation: (1) compute the anchor states and construct V from the desired derivatives; (2) form K_ij = G_0(z_i,z_j); (3) symmetrize K as (K + K^T)/2; (4) solve (K + lambda I) A = V using a Cholesky or QR solve; and (5) evaluate F_interp(t,y) = sum_i A_i G_0(y,z_i), returning f_theta(t,y) + F_interp(t,y). The paper's exact interpolation is the lambda = 0 case with nonsingular K. Use a small ridge lambda as a numerical safeguard and log the smallest eigenvalue and condition number of K. If conditioning is poor, merge similar anchors or increase lambda. First test on two-dimensional spiral and sine trajectories, then on a two-task MNIST latent ODE. Compare against a shared neural ODE trained only by backpropagation at equal parameter count and equal wall-clock budget. Measure anchor derivative error, final target-state error, gradient norm variance, and task accuracy. Success means near-zero simultaneous anchor error, fewer optimization steps, or better two-task accuracy without increasing inference cost excessively.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.