△ Mechanism confirmed, baseline not beaten 2026

Moment-preserving HT compression

Implementation & benchmark of arXiv:2607.01381 — A Local Macroscopic Conservative (LoMaC) low rank tensor method for the Vlasov-Maxwell system

Usefulness6/10

Difficulty6/10

Novelty7/10

Memory Memory Stability Accuracy Mlp Ssm World model Inference speedup Tensor decomposition Linear algebra Numerical analysis

Source paper: A Local Macroscopic Conservative (LoMaC) low rank tensor method for the Vlasov-Maxwell system arXiv:2607.01381 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Add a conservative correction after low-rank tensor compression so selected linear moments of an activation or learned state are exactly preserved. This can reduce tensor rank and memory without allowing compression error to accumulate in physically meaningful global quantities.

Formulas

$$\left(\begin{array}{c}{\bf J}_{1}\\ {\bf J}_{2}\\ {\bm{\kappa}}\end{array}\right)=\sum_{l_{1}}\sum_{l_{23}}\bm{\Theta}^{(1,2,3)}_{l_{1},l_{23}}\left\langle{\bf U}_{l_{23}}^{(2,3)},\left(\begin{array}{c}{\bf 1}_{v_{1}}\otimes{\bf 1}_{v_{2}}\\ {\bf v}_{1}\otimes{\bf 1}_{v_{2}}\\ {\bf 1}_{v_{1}}\otimes{\bf v}_{2}\\ \frac{1}{2}{\bf v}_{1}^{2}\otimes{\bf 1}_{v_{2}}+\frac{1}{2}{\bf 1}_{v_{1}}\otimes{\bf v}_{2}^{2}\end{array}\right)\right\rangle{\bf U}_{l_{1}}^{(1)}$$

$${f}^{n+1,*}=\frac{1}{4}{f}^{n-2}+\frac{3}{4}{f}^{n}-\frac{3}{2}\Delta t\left(v_{1}\partial_{x_{1}}({f}^{n})+E^{n}_{1}\partial_{v_{1}}({f}^{n})+v_{2}B^{n}_{3}\partial_{v_{1}}({f}^{n})+E^{n}_{2}\partial_{v_{2}}({f}^{n})-v_{1}B^{n}_{3}\partial_{v_{2}}({f}^{n})\right)$$

$$X_{\mathrm{cons}}=\widetilde X+\Phi G^{-1}\bigl(M(X)-M(\widetilde X)\bigr),\qquad G=\Phi^{T}W\Phi,\qquad M(X)=\Phi^{T}WX$$

$$M_i(X_{\mathrm{cons}})=M_i(X)\quad\text{for every }i,\qquad \|X_{\mathrm{cons}}-\widetilde X\|_{W}=\min_{Y:\,M(Y)=M(X)}\|Y-\widetilde X\|_{W}$$

Mathematical statement

The paper's LoMaC procedure first computes an intermediate tensor, projects it onto a subspace $\mathcal N$ whose basis spans prescribed macroscopic moments, and only then applies low-rank truncation. The displayed moment vector is $m(v_1,v_2)=(1,v_1,v_2,\tfrac12v_1^2+\tfrac12v_2^2)^T$, represented in tensor-product form as $(\mathbf 1_{v_1}\otimes\mathbf 1_{v_2},\mathbf v_1\otimes\mathbf 1_{v_2},\mathbf 1_{v_1}\otimes\mathbf v_2,\tfrac12\mathbf v_1^2\otimes\mathbf 1_{v_2}+\tfrac12\mathbf 1_{v_1}\otimes\mathbf v_2^2)^T$. For a neural tensor $X$ indexed by feature or state coordinates $z=(z_1,z_2)$, define moment functionals $M_i(X)=\langle X,\phi_i\rangle_w$, where $\phi_i$ are chosen basis tensors and $\langle A,B\rangle_w=\sum_z w_zA_zB_z$ is a weighted inner product. Given compressed $\widetilde X$, the smallest weighted-norm correction preserving the original moments is $X_{\mathrm{cons}}=\widetilde X+\Phi G^{-1}(M(X)-M(\widetilde X))$, where $\Phi=[\phi_1,\ldots,\phi_k]$, $G_{ij}=\langle\phi_i,\phi_j\rangle_w$, and $M(X)=(M_1(X),\ldots,M_k(X))^T$. This correction is the finite-dimensional orthogonal projection onto the affine set of tensors with the target moments.

Implementation notes

Integrate this at the output of a tensorized MLP, tensorized SSM state update, or world-model transition before the tensor is stored or passed to the next recurrent step. Let the state be a tensor $X\in\mathbb R^{d_1\times d_2\times\cdots}$, with two designated axes interpreted as coordinates or feature groups. First compute a rank-$r$ Tucker or hierarchical-Tucker approximation $\widetilde X=\operatorname{HTRound}(X,r)$. Select $k$ fixed basis tensors $\phi_i$: at minimum an all-ones tensor, coordinate tensors $z_1$ and $z_2$, and optionally the quadratic energy tensor $(z_1^2+z_2^2)/2$. Flatten the constrained axes, or contract the remaining axes, to compute the moments. Pseudocode: target = contract(Phi, X, weights); Xhat = HTRound(X, rank); current = contract(Phi, Xhat, weights); delta = target - current; G = contract(Phi, Phi, weights); coeff = solve(G + eps*I, delta); Xcons = Xhat + sum_i coeff[i]*Phi[i]. Here $W$ is the diagonal tensor of quadrature or feature weights; use uniform weights for ordinary activations. Compute the original moments once before compression, and calculate the small Gram matrix $G$ exactly. To retain a compact representation, store the correction as a rank-$k$ sum and optionally perform a second HT rounding with a tighter tolerance. Start with a small tensorized MLP on a synthetic advection or harmonic-oscillator rollout, comparing ordinary HT rounding at equal rank against conservative rounding. Measure rank, activation memory, one-step moment error, and 100- to 1000-step rollout drift. Success means at least 2x lower state memory at matched task loss, moments preserved to numerical precision, and materially lower long-horizon invariant drift. Use ordinary image classification as a negative control to determine whether the benefit depends on meaningful conserved statistics.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a self-contained moment-preserving low-rank compression experiment in moment_ht_experiment.py. The projection preserved all four moments to about 1.7e-12 over 200 recurrent steps, while ordinary rank-2 compression accumulated moment drift of 5.55 in L2; the minimum-norm sanity check also passed. However, at equal rank the correction increased the factor-storage upper bound from 130 to 390 units and slightly worsened state error, so this demonstrates the stability/conservation benefit but not a memory win.

Agent confidence: 9/10

Baseline: Rank-2 SVD: final/max moment drift 5.5521/5.6136 L2; mean relative state error 0.30415; factor storage 130 units.
Idea: Rank-2 SVD plus four-moment correction: final/max moment drift 1.262e-12/1.673e-12 L2; mean relative state error 0.30760; factor-storage upper bound 390 units.

Limitations: Only a small 32x32 synthetic recurrent system was tested, using matrix SVD as a 2D Tucker analogue rather than a full hierarchical-Tucker implementation. No task-loss benchmark, GPU test, runtime benchmark, second-rank sweep, or recompression of the correction back to a compact rank was performed; the reported idea storage is an upper bound for storing the correction as four additional rank-one terms.

How to run: python3 moment_ht_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.01381) Sep 3, 2026

No significant difference vs. the tuned baseline (+12.5%, p=0.4058, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0068
Idea mean: 0.0076
Effect (Δ): 0.0008 (+12.5%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.4058 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±34.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested; no memory or runtime advantage was measured, and the implementation compresses each 64-dimensional GRU hidden state as an 8x8 matrix rather than a full HT tensor. The benchmark used 8 epochs and 400 training/400 test examples per seed.

How to run: python3 bench_moment_preserving.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented moment-preserving rank-2 hidden-state compression as an end-to-end modification of the shared rnn_small architecture on the structurally matched dynamics track. Numerical and trained-model checks confirmed four hidden moments were preserved to about 1.4e-6 and 1.3e-6, but the idea had higher test MSE than the tuned baseline (0.007600 vs 0.006754), with paired delta +0.000846 and permutation p=0.4058; therefore it did not produce a significant win.

Baseline: Dynamics rnn_small, 8 paired seeds, lr sweep {0.001, 0.003, 0.006}, best lr=0.006; mean test MSE 0.0067540, std 0.0028601.
Idea: Moment-preserving rank-2 hidden compression, same 8 seeds and learning-rate union; mean test MSE 0.0075996, std 0.0019136. Paired delta +0.0008456, 3/8 wins, permutation p=0.4058.
Mechanism signature: Trained-model hidden-state moment residual 1.3113e-6; compression relative perturbation 0.4047; confirmed=true.

Benchmark result