Gradient-Commutator Neural Dynamics
Implementation & benchmark of arXiv:2607.26890 — The Lie algebra generated by gradient vector fields
Source paper: The Lie algebra generated by gradient vector fields arXiv:2607.26890 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Build a continuous-time neural dynamics module from scalar potential networks and their iterated Lie brackets instead of directly predicting an unrestricted vector field. Gradient primitives provide structured vector fields, while commutators add non-conservative and rotational directions; the paper proves that finite spans of such objects generate every smooth vector field on the stated compact manifold.
Formulas
Mathematical statement
For a smooth manifold M with smooth non-degenerate symmetric bilinear form g, the metric gradient of f is defined by df_x(v)=g_x(
abla_g f(x),v) for every v\in T_xM. The paper defines \mathfrak L_g^0=\{
abla_g f:f\in C^\infty(M,\mathbb R)\} and recursively \mathfrak L_g^m=\{X+[
abla_g f,Y]:f\in C^\infty(M,\mathbb R),X,Y\in\mathfrak L_g^{m-1}\}. Its main result states that every smooth vector field is a finite linear combination of iterated Lie brackets of gradient fields. In Euclidean coordinates with g=I, u_f(x)=
abla f(x), and the Lie bracket is [u,v](x)=Dv(x)u(x)-Du(x)v(x), where Du and Dv are Jacobians. Parameterize a neural field as F_ heta(x)=\sum_j a_jB_j(x), where each B_j is a primitive gradient or a recursively formed bracket. A derivative-free flow construction uses the commutator identity \Phi_{-\epsilon}^v\circ\Phi_{-\epsilon}^u\circ\Phi_{\epsilon}^v\circ\Phi_{\epsilon}^u(x)=x+\epsilon^2[u,v](x)+O(\epsilon^3), with \Phi_t^u the time-t flow of u.
Implementation notes
Integrate the construction at the vector-field evaluation point of a neural ODE, continuous-depth residual block, or latent state-space/world-model transition. Begin with Euclidean states, so g=I and each primitive field is u_i(x)=grad_x f_i(x), where f_i is a small scalar MLP. Select K=1 or K=2 bracket terms and learn scalar coefficients; for example, F(x)=a_1u_1(x)+a_2u_2(x)+a_3[u_1,u_2](x)+a_4[u_2,[u_1,u_3]](x). Direct-autodiff pseudocode is: input x; evaluate scalar potentials f_i(x); compute u_i=autograd.grad(sum(f_i),x,create_graph=True); compute each bracket with Jacobian-vector products as bracket(u,v)=JVP(v,u)-JVP(u,v); recursively construct the selected terms; return dx/dt=F(x). Avoid materializing full Jacobians. As an alternative, approximate a bracket by four small Euler or RK flow steps: x1=x+epsilon*u(x), x2=x1+epsilon*v(x1), x3=x2-epsilon*u(x2), x4=x3-epsilon*v(x3), and use (x4-x)/epsilon^2 as the bracket residual. Train potential weights and coefficients jointly with the task objective. The formulas and bracket computation are exact; the theorem supplies expressivity motivation but no usable approximation constant, so do not claim a guaranteed depth. First test on 2D and 4D spiral, rotation, and non-conservative dynamical-system prediction tasks, comparing pure-gradient, bracket-structured, and unconstrained MLP vector fields at equal parameters and equal FLOPs. Record trajectory error, long-horizon rollout stability, Jacobian eigenvalues, gradient norms, and wall-clock cost. Success means the structured model fits rotational systems much better than pure gradients while matching or exceeding the long-horizon stability of the unconstrained MLP.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.