✓✓ Beats tuned baseline 2026

Kolmogorov-Lie Unitary Layer

Implementation & benchmark of arXiv:2607.03187 — Quantum Kolmogorov--Arnold representation theorem for continuous unitary-valued maps

Usefulness6/10

Difficulty6/10

Novelty7/10

Architecture Accuracy Stability Scalability Mlp Transformer Ssm Architecture Lie groups Representation theory Algebra Differential geometry

Source paper: Quantum Kolmogorov--Arnold representation theorem for continuous unitary-valued maps arXiv:2607.03187 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.

Formulas

$$\mathfrak{u}(n)=\{A\in\mathbb{C}^{n\times n}:A^*=-A\},\qquad \mathcal{U}(n)=\{U\in\mathbb{C}^{n\times n}:U^*U=I\},\qquad U(x)=\exp(A(x)).$$

$$A(x)=\sum_{k=1}^{K}G_k\!\left(s_k(x)\right),\qquad s_k(x)=\sum_{j=1}^{d}\psi_{kj}(x_j),\qquad G_k(t)^*=-G_k(t).$$

$$U(x)=\prod_{k=1}^{K}\exp\!\left(G_k(s_k(x))\right),\qquad \left(\prod_{k=1}^{K}M_k\right)=M_K\cdots M_2M_1.$$

$$G_k(t)=V_k(t)-V_k(t)^*,\qquad U(x)^*U(x)=I\ \text{for every }x.$

Mathematical statement

The paper's local theorem concerns a continuous map U:X o\mathcal{U}(n) whose values lie in a neighborhood of the identity, where \mathcal{U}(n)=\{U\in\mathbb{C}^{n imes n}:U^*U=I\}. In that neighborhood the principal matrix logarithm exists, so U(x)=\exp(A(x)) with A(x)^*=-A(x) anti-Hermitian. The additive version has the Kolmogorov-Arnold form A(x)=\sum_{k=1}^{K}G_k(s_k(x)), where s_k(x)=\sum_{j=1}^{d}\psi_{kj}(x_j), x=(x_1,\ldots,x_d), \psi_{kj}:\mathbb{R} o\mathbb{R} are continuous univariate inner functions, and G_k:\mathbb{R} o\mathfrak{u}(n) are continuous anti-Hermitian-valued outer functions; \mathfrak{u}(n)=\{A:A^*=-A\}. Because matrix exponentials generally do not commute, the factorized form is U(x)=\prod_{k=1}^{K}\exp(G_k(s_k(x))) with a fixed product order. The ML adaptation uses low-rank anti-Hermitian generators G_k(t)=V_k(t)-V_k(t)^*, or scalar combinations of fixed anti-Hermitian generators, with each scalar coefficient computed by a spline or small MLP. The local-neighborhood assumption is enforced by bounding generator norms or using short products, rather than claiming a global representation on all of \mathcal{U}(n).

Implementation notes

Integrate the layer where a model currently applies an input-conditioned dense linear map or a learned unitary or orthogonal transformation. Use a complex feature vector h\in\mathbb{C}^n for the first prototype; a real-valued version can replace anti-Hermitian matrices by real skew-symmetric matrices. Choose K=4--16 factors. Define each inner function \psi_{kj} as a cubic spline with 8--32 knots, or as a tiny two-layer MLP that receives only normalized coordinate x_j. Compute s_k=\sum_j\psi_{kj}(x_j). For the simplest parameterization, let each factor use a fixed learned anti-Hermitian matrix H_k and scalar coefficient a_k(s_k), giving G_k(s_k)=a_k(s_k)H_k. For a more expressive but still structured version, use G_k(s_k)=a_k(s_k)H_k+b_k(s_k)J_k, where H_k^*=-H_k and J_k^*=-J_k. Apply the factors sequentially: normalize x; for k=1,...,K, compute s_k; compute the bounded coefficients; form G_k; update h\leftarrow\exp(G_k)h; return h. Initialize spline outputs and generator scales near zero so the layer starts close to identity. Bound coefficients with tanh and rescale each generator so \|G_k\|_2\leq r, preventing the product from leaving the theorem's local regime. The exact mathematical property computed from the construction is unitarity, since exp(G_k)^*exp(G_k)=I for anti-Hermitian G_k and products of unitary matrices remain unitary. Estimate runtime empirically; start with dense matrix exponentials and then test the Cayley approximation \exp(G)\approx(I-G/2)^{-1}(I+G/2), which is exactly unitary for skew-Hermitian G up to numerical solve error. First run a four-layer complex MLP on CIFAR-10 or a synthetic long-range sequence task, comparing this layer against a dense input-conditioned controller and a standard Householder/unitary layer at matched parameter count. Record validation accuracy, training loss versus wall-clock and FLOPs, gradient-norm variance, activation-norm drift, and the residual \|U^*U-I\|_F. A successful result is equal-or-better accuracy with substantially fewer controller parameters, stable norms through long compositions, and faster loss descent or lower wall-clock cost.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built the Kolmogorov-Lie Unitary Layer with separable coordinate MLPs, bounded scalar coefficients, anti-Hermitian generators, and sequential matrix exponentials. The mathematical claim was reproduced: a 32-factor product had unitarity residual 1.31e-5 in float32, and the learned KLU model had residual 8.67e-7 with mean norm drift 2.12e-7. In the toy learning task KLU used 564 parameters versus 4,352 for the dense controller, but had higher test MSE (0.000522 versus 0.000374) and was slower (19.6s versus 13.3s), so the stability and parameter-efficiency effect is real but there was no accuracy or speed win.

Agent confidence: 8/10

Baseline: Dense controller: 4,352 parameters, test MSE 0.0003740, norm drift 0.009014, unitarity residual 0.5433, 13.26s.
Idea: KLU: 564 parameters, test MSE 0.0005223, norm drift 2.12e-7, unitarity residual 8.67e-7, 19.61s. Direct math check: single exponential residual 5.97e-6; 32-factor product residual 1.31e-5.

Limitations: Only a small synthetic complex-vector regression task was tested; no CIFAR-10, sequence benchmark, Cayley approximation, FLOP-normalized comparison, Householder baseline, multiple random seeds, or large-scale timing study was run. The dense baseline was intentionally unconstrained and therefore not a unitary comparator.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.03187) Sep 4, 2026

Beats the tuned baseline by 54.5% (p=0.0156, wins 7 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0119
Idea mean: 0.0054
Effect (Δ): -0.0065 (−54.5%; negative = idea better)
Wins: 7 / 8 paired seeds
p-value: 0.0156 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±37.6%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested; CIFAR-10, sequence, tabular, Householder layers, Cayley approximations, FLOP-normalized timing, and larger hidden sizes were not evaluated. The prototype uses real anti-symmetric generators as the real-valued analogue of anti-Hermitian complex generators.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Worked ✓

Agent confidence: 8/10

Implemented the Kolmogorov-Lie unitary recurrent layer as a matched modification of a small dynamics RNN, using four input-conditioned anti-symmetric exponential factors and the benchmark train_model path. On the structurally appropriate pendulum-control dynamics track, it achieved lower test MSE than the dense recurrent baseline across 7/8 paired seeds: 0.005406 versus 0.011876, paired delta -0.006470, permutation p=0.0156. The trained-model mechanism signature confirmed near-unitarity with residual 1.82e-6, so both the predicted stability property and a significant task-metric improvement were observed.

Baseline: Dense recurrent baseline, best lr=0.01, 8-seed mean test MSE 0.0118761, std 0.0048702.
Idea: KLU recurrent layer, best lr=0.01, 8-seed mean test MSE 0.00540594, std 0.00174744; 7/8 paired wins; permutation p=0.0156; observed unitarity residual 1.8216e-6.

Benchmark result