△ Mechanism confirmed, baseline not beaten 2026

Moment-Preserving Anisotropic Feature Tree

Implementation & benchmark of arXiv:2607.04881 — Towards Fully Dynamic Omnitrees: Moment-Conserving Anisotropic Compression With Wavelets

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Memory Accuracy Scalability Mlp Embedding World model Memory Harmonic analysis Wavelet Geometry Linear algebra

Source paper: Towards Fully Dynamic Omnitrees: Moment-Conserving Anisotropic Compression With Wavelets arXiv:2607.04881 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a dense multiresolution voxel or hash-grid encoder with an omnitree-like anisotropic feature partition. Each cell stores a vector-valued scaling feature and its children are introduced only when local Haar detail energy is large; coarsening replaces children by their mean, so compression does not introduce an arbitrary offset. Splitting can be restricted to the coordinate whose one-dimensional detail coefficient is largest, allowing thin structures to receive resolution only in the relevant directions.

Formulas

$$f(\mathbf{x})=\sum_{v:\,\vec b(v)=\vec 0}s^v\,\phi^v(\mathbf{x}),\qquad \phi^v(\mathbf{x})=\begin{cases}1,&\mathbf{x}\in\operatorname{rect}(v),\\0,&\text{else.}\end{cases}$$

$$\begin{pmatrix}w_{\emptyset}\\w_{\{x\}}\\w_{\{y\}}\\w_{\{x,y\}}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1&1&1\\1&1&-1&-1\\1&-1&1&-1\\1&-1&-1&1\end{pmatrix}\begin{pmatrix}c_{00}\\c_{01}\\c_{10}\\c_{11}\end{pmatrix},\qquad c_{ij}\in\mathbb{R}^{C}.$$

$$c_{\mathrm{parent}}=w_{\emptyset}=\frac{1}{2^d}\sum_{i=1}^{2^d}c_i,\qquad E_{\mathrm{detail}}=\sum_{k\in\{0,1\}^{d}\setminus\{\mathbf{0}\}}\lVert w_k\rVert_2^2.$$

$$j^*=\arg\max_{j\in\{1,\ldots,d\}}\lVert w_{\{j\}}\rVert_2,\qquad \text{split cell along coordinate }j^*\text{ if }E_{\mathrm{detail}}>\tau.$$

Mathematical statement

The paper represents a piecewise-constant function on leaf rectangles as f(x)=sum_v s^v phi^v(x), where v ranges over leaves, s^v is the value or feature attached to leaf v, and phi^v(x)=1 on rect(v) and 0 elsewhere. For a parent with 2^d equal children, the tensor-product Haar transform maps child values to one scaling coefficient and 2^d-1 detail coefficients. In two dimensions the transform is based on the Hadamard matrix H=[[1,1,1,1],[1,1,-1,-1],[1,-1,1,-1],[1,-1,-1,1]]; with normalized rows, the first output is the child mean and the other three outputs are horizontal, vertical, and diagonal details. For vector-valued features, apply the same transform independently to every feature channel. If c_i are the child feature vectors and m=2^d, the moment-preserving coarsening rule is c_parent=(1/m) sum_i c_i. Because the Haar basis is orthogonal, the squared error from dropping details is exactly proportional to E_detail=sum_{k != emptyset} ||w_k||_2^2. For anisotropic splitting, choose the coordinate with the largest directional detail norm, j*=argmax_j ||w_{j}||_2, and split only along that coordinate when E_detail exceeds a threshold.

Implementation notes

Integrate this at the spatial feature encoder, before the MLP or attention block that consumes coordinates. Start with a 2D coordinate MLP or a 3D NeRF-style voxel encoder: every active rectangular cell stores a C-dimensional feature vector and integer dyadic bounds, and the decoder is an unchanged small MLP receiving the selected cell feature together with the normalized coordinate. Maintain a priority queue keyed by detail energy. Pseudocode: initialize a root cell; for every candidate cell, accumulate or sample the feature vectors c_i in its possible child bins; compute the channelwise Haar outputs using the displayed matrix; calculate E_detail; if E_detail exceeds tau_split, create two children along j*=argmax_j ||w_{j}||_2 or all 2^d children; initialize child features by inverse Haar reconstruction; after every K optimizer steps, inspect sibling groups and merge them when E_detail<tau_merge and a held-out loss estimate increases by less than delta; assign the merged feature exactly as c_parent=(1/m)sum_i c_i. Compute the Haar transform and merge rule exactly. Estimate child features from current learned embeddings, sampled decoder targets, or averages of local residual-correcting features. Calibrate thresholds on a held-out set and freeze topology between adaptation events. The first cheap experiment should train a small coordinate MLP on MNIST images or a 3D NeRF scene, comparing a dense grid, isotropic octree, and adaptive anisotropic tree at equal parameter memory. Record PSNR or accuracy versus active-cell count, peak memory, decoder evaluations, and split/merge frequency. Success means at least 2x fewer active cells at equal PSNR, or higher PSNR at equal memory, without unstable topology oscillation.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a deterministic 2D moment-preserving anisotropic feature-tree MVP with exact Haar transforms, mean coarsening, detail-energy thresholding, and directional splitting. Parseval matched across 36 channel-scale cases with relative gaps below 3.1e-16, directional selection switched at the predicted equal-detail crossing, and larger thresholds reduced cells while increasing reconstruction error. The toy anisotropic reconstruction also outperformed isotropic and dense baselines at all tested budgets, but this is not evidence of a trained neural-network win.

Agent confidence: 8/10

Baseline: Isotropic MSE at budgets 16/32/64/128: 0.019684, 0.017850, 0.016779, 0.012726. Dense-grid MSE: 0.019636, 0.020298, 0.017120, 0.012112.
Idea: Anisotropic MSE at budgets 16/32/64/128: 0.008410, 0.008224, 0.008174, 0.008166, using 15/31/63/127 cells. Parseval relative gap was below 3.1e-16.

Limitations: This is a CPU toy piecewise-constant reconstruction experiment, not a learned coordinate MLP, NeRF, or attention encoder. It does not test optimizer dynamics, trained feature channels, held-out merge decisions, topology oscillation, runtime, peak memory, or a genuine equal-parameter neural baseline.

How to run: python3 experiment.py > results.json

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.04881) Sep 2, 2026

Significantly WORSE than the tuned baseline (+29.5%, p=0.0169, wins 1 / 8 paired seeds).

Benchmark: Geometry support regression
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.0231
Idea mean: 0.0299
Effect (Δ): 0.0068 (+29.5%; negative = idea better)
Wins: 1 / 8 paired seeds
p-value: 0.0169 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±22.5%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Tested one structurally matched 2-D geometry/PDE-like regression track and one dense-grid baseline representation; did not test 3-D NeRF, isotropic octree, dynamic split/merge events, larger feature/channel budgets, alternative tree initialization, or runtime/peak-memory scaling.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented a moment-preserving anisotropic Haar-priority feature tree and compared it end-to-end against a dense 8×8 learned feature grid with the same decoder, optimizer, epochs, batch size, and shared learning-rate sweep. Haar Parseval error was 2.72e-16, and the trained-model signature was confirmed with detail/coarsening correlation 0.721. However, the idea was significantly worse: test MSE 0.029899 versus baseline 0.023088, paired delta +0.006811, permutation p=0.0169.

Baseline: geometry_support_regression / mlp_tiny; best lr=0.01; full 8-seed MSE mean 0.0230883560, std 0.0053176263; sweep means lr 0.001/0.003/0.01 = 0.04883256/0.02276584/0.02171619
Idea: best shared lr=0.01; full 8-seed MSE mean 0.0298992444, std 0.0063932703; sweep means lr 0.001/0.003/0.01 = 0.05844058/0.04404880/0.03228097
Math check: Haar mean max error 0; relative Parseval error 2.724e-16
Mechanism signature: 27 trained sibling groups; detail-energy versus decoder-coarsening MSE correlation 0.7206; observed/predicted mean ratio 0.3187; confirmed=true
Paired comparison: delta_mean=+0.0068108884 (positive is worse), idea_wins=1/8, p_value=0.0169, verdict=idea worse (significant)

Benchmark result