Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation

What the math gives to ML

The paper's transferable asset is a solver-level differentiation rule: when a forward computation is an ordered sequence of local implicit block solves, its exact discrete adjoint is obtained by reverse-order local adjoint solves rather than by constructing a global Jacobian or storing the entire forward trace. This preserves the gradient of the finite computation that actually ran, including incomplete convergence and sweep ordering, unlike equation-level implicit differentiation at a nominal fixed point. Neural networks with block-coordinate equilibrium layers, graph message-passing sweeps, learned iterative optimizers, and differentiable sparse inference can use the same structure. The most promising initial experiment is a block-implicit neural layer with a custom backward that stores only block states and solves small transposed local systems in reverse order.