# GLM Energy-Preserving Feature Block

- ID: 1266
- Canonical URL: https://synthcore.org/idea/1266/glm-energy-preserving-feature-block
- API JSON: https://synthcore.org/api/idea/1266.json
- API Markdown: https://synthcore.org/api/idea/1266.md
- Verification status: unverified
- Source: [arXiv:2607.22151](https://arxiv.org/abs/2607.22151)
- Category: dynamics
- Solves: stability, accuracy, scalability
- ML areas: cnn, ssm, training-dynamics
- Math tags: dynamical-systems, linear-algebra, pde, symplectic
- Ratings: usefulness 5/10; difficulty 6/10; novelty 5/10

## Idea description

Replace an unconstrained residual block by a four-field feature dynamics containing a primary feature T, flux-like auxiliary features J, curl-cleaning features psi, and a scalar cleaning feature phi. Couple these fields with learned skew-adjoint spatial operators so that the reversible block preserves the squared feature norm, while a separately controlled relaxation term can remove high-frequency or constraint-violating components. Use an exact Cayley update rather than explicit Euler to prevent norm growth across many layers.

## Mathematical statement

The reversible heat-GLM system is the four-field PDE $$\partial_t T+c_0\nabla\cdot\mathbf{J}=0,\quad \partial_t\mathbf{J}+c_0\nabla T+c_h\nabla\times\boldsymbol{\psi}=0,\quad \partial_t\boldsymbol{\psi}-c_h\nabla\times\mathbf{J}+c_h\nabla\varphi=0,\quad \partial_t\varphi+c_h\nabla\cdot\boldsymbol{\psi}=0.$$ Here T and phi are scalar fields, J and psi are d-dimensional vector fields, c_0 and c_h are propagation speeds, and divergence and curl are spatial differential operators. Its quadratic energy density is $$\mathcal{E}(T,\mathbf{J},\boldsymbol{\psi},\varphi)=\tfrac12(T^2+\|\mathbf{J}\|^2+\|\boldsymbol{\psi}\|^2+\varphi^2).$$ The paper writes the system as $$\partial_t u+\sum_i B_i\partial_i u=0,$$ with each field-coupling matrix B_i symmetric. For a neural discretization, choose learned operators A_i satisfying A_i^T=-A_i and define $$K=-\sum_i B_i\otimes A_i.$$ Because B_i^T=B_i and A_i^T=-A_i, K^T=-K, so the continuous linear flow preserves ||u||_2^2. The Cayley discretization $$u_{k+1}=\left(I-\frac{h}{2}K\right)^{-1}\left(I+\frac{h}{2}K\right)u_k$$ is norm-preserving for every step size h when the solve is exact. Optional relaxation uses a positive semidefinite R=R^T\succeq0 and applies $$u_{k+1}=\operatorname{Cayley}(hK)(I-h\lambda R)u_k,$$ with lambda >= 0.

## Key formulas

- $$\frac{\partial T}{\partial t}+c_{0}\,\nabla\cdot\mathbf{J}=0,\qquad \frac{\partial\mathbf{J}}{\partial t}+c_{0}\nabla T+c_{h}\nabla\times\boldsymbol{\psi}=0,$$
- $$\frac{\partial\boldsymbol{\psi}}{\partial t}-c_{h}\nabla\times\mathbf{J}+c_{h}\nabla\varphi=0,\qquad \frac{\partial\varphi}{\partial t}+c_{h}\nabla\cdot\boldsymbol{\psi}=0,$$
- $$\mathcal{E}(T,\mathbf{J},\boldsymbol{\psi},\varphi)=\frac{1}{2}\left(T^{2}+\|\mathbf{J}\|^{2}+\|\boldsymbol{\psi}\|^{2}+\varphi^{2}\right),$$
- $$u_{k+1}=\left(I-\frac{h}{2}K\right)^{-1}\left(I+\frac{h}{2}K\right)u_k,\qquad K=-\sum_i B_i\otimes A_i,\quad B_i^T=B_i,\quad A_i^T=-A_i.$$

## Implementation notes

Integrate this as a replacement for a residual CNN block or as one step of a continuous-depth state-space model. First maintain four feature groups at each spatial location: T with C channels, J with dC channels, psi with dC channels, and phi with C channels; concatenate them into u. Parameterize each spatial operator with a convolution W_i and enforce skew-adjointness by A_i=(W_i-W_i^T)/2, where W_i^T is the adjoint convolution under the selected padding and boundary convention. Construct small fixed or learned symmetric field-coupling matrices B_i implementing the c_0 gradient/divergence and c_h curl couplings, then compute K u=-sum_i(B_i tensor A_i)u. Use the Cayley update by solving (I-hK/2)v=(I+hK/2)u with several GMRES iterations, or use an exact dense/block solve when the channel dimension is small. For damping, apply v <- v-h lambda Rv with diagonal R that selectively damps psi and phi. The paper determines the symmetric coupling and energy invariant; learned operators, h, lambda, and R are empirical choices. Measure norm drift ||v||^2/||u||^2 and residual-solve error. The first experiment should compare a 32-layer CIFAR-10 CNN against standard residual, antisymmetric, and orthogonal/Cayley blocks at matched parameters and FLOPs, testing h in {0.25,0.5,1,2}. Record loss descent, gradient norms, accuracy, norm drift, and NaN rate. Success means stable training at h=1 or 2, energy drift below 1e-4 per reversible block, and equal-or-better accuracy than the baselines.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.