Idea description

Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension $d=\xi(G)$, whereas a discrete state encoding is lower-bounded by $N\geq\chi(G)$. This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed advantage can be measured directly.

Formulas

$$\hat{P}_{i}=\ket{v_i}\bra{v_i},\qquad (i,j)\in E\ \Longrightarrow\ \hat{P}_{i}\hat{P}_{j}=0\ \Longleftrightarrow\ \langle v_i,v_j\rangle=0.$$

$$\xi(G)=\min\left\{d:\exists\ \{|v_i\rangle\in\mathbb{R}^{d}\}_{i\in V},\ \|v_i\|_2=1,\ \langle v_i,v_j\rangle=0\ \forall(i,j)\in E\right\}.$$

$$N\geq\chi(G),\qquad d=\xi(G).$$

$$\mathcal{L}=\mathcal{L}_{\mathrm{task}}+\lambda_{\mathrm{orth}}\sum_{(i,j)\in E}\langle v_i,v_j\rangle^2+\lambda_{\mathrm{norm}}\sum_{i\in V}(\|v_i\|_2^2-1)^2.$$

Mathematical statement

The paper represents each graph vertex $v_i\in V$ by a rank-1 projector $\hat P_i=|v_i\rangle\langle v_i|$, with adjacent vertices mapped to orthogonal projectors. Here $G=(V,E)$ is an exclusivity graph, $E$ contains pairs of mutually exclusive events, and $|v_i\rangle\in\mathbb{R}^d$ or $\mathbb{C}^d$ is a unit vector. The orthogonal rank is $\xi(G)=\min\{d:\exists\{|v_i\rangle\}_{i\in V},\ \|v_i\|_2=1,\ \langle v_i,v_j\rangle=0\ \forall(i,j)\in E\}$. The classical memory requirement is $N\geq\chi(G)$, where $\chi(G)$ is the minimum number of colors assigning different colors to adjacent vertices; the quantum-style memory dimension is $d=\xi(G)$. The transferable property is that dense incompatibility constraints can require many discrete colors while admitting a much smaller continuous orthogonal representation. In a neural implementation, exact orthogonality is relaxed with an edge penalty, while the recurrent state update learns how to move between event vectors.

Implementation notes

1. Integration point: implement a GRU-like recurrent recognizer for graph-generated symbolic sequences. Let the alphabet contain one token per graph vertex, let $e_i\in\mathbb{R}^d$ be a learned event embedding, and maintain hidden state $h_t\in\mathbb{R}^d$. Normalize embeddings by $v_i=e_i/(\|e_i\|_2+\epsilon)$. Feed the token and previous state into a gated transition, using the same dimension $d$ for the hidden memory rather than a one-hot state table. 2. Pseudocode: initialize $e_i$ and recurrent parameters; for each minibatch sequence $x_{1:T}$, compute $v_i=e_i/(\|e_i\|_2+\epsilon)$, set $h_0=0$, and for $t=1,\ldots,T$ compute $h_t=\mathrm{GRU}(v_{x_t},h_{t-1})$, then predict the accept/reject label from $h_t$. Add $\lambda_{\mathrm{orth}}\sum_{(i,j)\in E}(v_i^\top v_j)^2$ and $\lambda_{\mathrm{norm}}\sum_i(\|e_i\|_2^2-1)^2$ to cross-entropy. Optionally project the hidden state after each step onto the span of active event vectors. 3. Mathematics versus estimation: compute $\chi(G)$ for small graphs with an integer-programming coloring solver. Estimate $\xi(G)$ by optimizing the orthogonality objective over random embeddings for candidate dimensions $d=1,2,\ldots$; the smallest dimension with near-zero edge violation is the empirical orthogonal rank. The network does not receive these values; they determine the width sweep and predicted classical/continuous memory comparison. Use real vectors first, then test complex-valued embeddings only if needed. 4. First experiment: generate promise languages from graph walks, where a sequence is a valid walk under graph-dependent transitions and the label depends on whether its final event is compatible with a designated start context. Compare a discrete-state RNN with width $\chi(G)$, a standard GRU with width $\xi(G)$, and the same GRU without the orthogonality penalty. Use 5--10 small graphs, including odd cycles and Boolean-orthogonality-derived graphs, sequence lengths 32--256, and equal parameter/FLOP budgets. Success means matching accuracy with substantially smaller hidden dimension, retaining accuracy on longer sequences, and showing lower validation loss or fewer parameters at matched accuracy. Report the Gram edge violation $\max_{(i,j)\in E}|v_i^\top v_j|$ to verify that gains come from the intended structure rather than ordinary embedding capacity.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a reproducible orthogonal-rank memory MVP and verified the core graph claim: the 13-vertex graph has chi(G)=4 while an explicit 3D real orthogonal representation exists, with a 3-clique proving xi(G)>=3. The width-3 GRU used 35% fewer parameters than width 4 and matched its toy-task accuracy (72.3%), while the orthogonality penalty reduced edge violation from 0.99 to 0.007. However, the task was weak and the penalty did not improve accuracy or loss, so this is evidence for structural compression rather than a demonstrated neural performance win.

Agent confidence: 7/10

Baseline: Width-4 GRU: 177 parameters, long-sequence accuracy 0.7230, loss 0.5884, edge violation 0.9751.
Idea: Width-3 orthogonal GRU: 115 parameters, long-sequence accuracy 0.7230, loss 0.5895, edge violation 0.0071; graph check chi=4, xi=3.

Limitations: Only one 13-vertex graph and one simple compatibility task were tested; the task has class imbalance and observed accuracy is close to the majority-class baseline. No longer sequence sweep, multiple seeds, FLOP-matched training, exact integer-programming coloring, or complex-valued representations were tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.00507) Sep 3, 2026

Significantly WORSE than the tuned baseline (+7.2%, p=0.0169, wins 1 / 8 paired seeds).

Benchmark: Sparse_cycle_compatibility
Model: custom_gru
Paired seeds: 8
Baseline mean: 0.5538
Idea mean: 0.5938
Effect (Δ): 0.04 (+7.2%; negative = idea better)
Wins: 1 / 8 paired seeds
p-value: 0.0169 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±3.8%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

The registered track provides one-event categorical inputs, so the experiment used duplicated two-step sequences and did not test long-range contextual retention. The custom GRU was local rather than a built-in model, although both systems shared it exactly. The mechanism signature lacks a baseline trained-model embedding measurement, so it cannot claim quantitative confirmation; lambda=0 was selected as the best idea setting, meaning the orthogonality intervention itself did not win.

How to run: python3 bench_orthogonal_memory.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Failed ✗

Agent confidence: 10/10

The idea was tested on the registered sparse_cycle_compatibility graph-context classification track with identical GRU systems, matched batch/epochs, and a shared learning-rate union. The best orthogonal-memory result was worse than the tuned baseline (delta_mean +0.0400, p=0.0169), so the required significant-win criterion was not met; the mechanism signature is also unconfirmed because the baseline trained-model edge violation was not captured.

Baseline: mean err 0.55375 over 8 seeds; best lr=0.01; per-seed [0.6650, 0.5750, 0.6300, 0.3300, 0.5200, 0.5400, 0.6350, 0.5350]
Idea: best mean err 0.59375 over 8 seeds at lr=0.01, lambda_orth=0.0; per-seed [0.7300, 0.6100, 0.6900, 0.3700, 0.5700, 0.5950, 0.6300, 0.5500]; delta +0.0400; idea wins 1/8

Benchmark result