Casimir-Preserving Matrix Optimizer
Implementation & benchmark of arXiv:2607.18312 — Vakonomic Fluids
Source paper: Vakonomic Fluids arXiv:2607.18312 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Introduce an auxiliary matrix-valued optimizer state whose update is a Lie–Poisson flow discretized by similarity transforms rather than additive Euler steps. Because similarity transforms preserve $\operatorname{tr}(Z^k)$ and the full eigenvalue multiset, long training runs avoid spectral drift in the optimizer state; the state can then generate a preconditioned update for ordinary neural-network parameters.
Formulas
Mathematical statement
The paper studies constrained geodesic dynamics whose discrete vakonomic trajectories are Lie–Poisson and preserve Casimirs of the form $C_k(Z)=\operatorname{tr}(Z^k)$. Here $Z\in\mathbb{R}^{d\times d}$ is an auxiliary state, $k\geq1$ is an integer, and a Casimir is a quantity constant along the Lie–Poisson orbit. The continuous matrix Lie–Poisson equation has the commutator form $\dot Z=[Z,\nabla_Z H(Z)]$, where $H$ is a scalar energy and $[A,B]=AB-BA$. A discrete conjugation step $Z_{t+1}=Q_tZ_tQ_t^{-1}$ with invertible $Q_t$ preserves every trace power exactly: $\operatorname{tr}(Z_{t+1}^k)=\operatorname{tr}(Z_t^k)$ by cyclicity of trace. For numerical stability choose $Q_t=\exp(\eta K_t)$ with skew-symmetric $K_t^\top=-K_t$, so $Q_t$ is orthogonal and the eigenvalues and Frobenius norm of $Z$ are preserved. Use $H_t(Z)=\tfrac12\lVert Z-G_t\rVert_F^2$ or a low-rank approximation of the current gradient matrix $G_t$ to define $K_t=Z_tG_t^\top-G_tZ_t^\top$, with $\eta$ a small step size. The neural-network parameter update is generated separately from $Z_t$, for example $\theta_{t+1}=\theta_t-\alpha\,\mathcal{P}(Z_t)\,g_t$, where $g_t=\nabla_\theta L(\theta_t)$ and $\mathcal{P}(Z_t)$ is a positive diagonal or blockwise preconditioner extracted from $Z_t^\top Z_t+\epsilon I$.
Implementation notes
Integrate this at the optimizer level, not inside the forward network. Partition parameters into blocks of size $d\times d$; for each block maintain an auxiliary dense or low-rank matrix $Z_t$ and use the block’s flattened gradient $g_t$ to form a gradient matrix $G_t$ by reshaping or by a fixed learned linear projection. At step $t$, compute $g_t=\nabla_\theta L(\theta_t)$, build $G_t$, and set $K_t=Z_tG_t^\top-G_tZ_t^\top$. For a cheap MVP use the first-order orthogonal approximation $Q_t\approx\operatorname{Cay}(\eta K_t)=(I-\eta K_t/2)^{-1}(I+\eta K_t/2)$; update $Z_{t+1}=Q_tZ_tQ_t^\top$. Compute the parameter preconditioner $\mathcal P(Z_t)$ from the diagonal of $Z_t^\top Z_t$ and apply $\theta_{t+1}=\theta_t-\alpha\mathcal P(Z_t)g_t$, optionally with decoupled weight decay. The exact mathematical quantity is the conjugacy-preserved trace spectrum; estimate empirical optimizer quality using loss, gradient norms, and trace errors $|\operatorname{tr}(Z_t^k)-\operatorname{tr}(Z_0^k)|$ for $k=1,2,3$. For scalability, replace dense matrices by a rank-$r$ factorization $Z=UV^\top$ and transform $U\leftarrow QU$, $V\leftarrow QV$. First test on a 2-layer MLP and ResNet-18 on CIFAR-10, comparing SGD-momentum, AdamW, and this optimizer at matched update FLOPs. Use $d=16$ or $32$, $r=4$ or $8$, and sweep $\eta\in\{10^{-3},10^{-2},10^{-1}\}$. Success means lower variance of training loss and gradient norms over long runs, no optimizer-state spectral drift, and equal-or-better validation accuracy at the same number of optimizer steps. A negative result—higher cost without improved stability or accuracy—should reject the approach.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.