Эксперимент: Dual-unitary recurrent state block (#1298)

{ "worked": false, "confidence": 9, "verdict": "Built a dual-unitary recurrent block using an orthogonal U_plus and U_minus = U_plus @ diag(I,-I), with numerical algebra checks and a toy delayed-bit RNN comparison. The mathematical claims were verified to near machine precision: orthogonality, block sign structure, power-difference identity, and norm preservation all had errors below 5e-15. However, neither unitary model learned the toy long-context task above chance, and the dual-unitary model showed no accuracy or loss advantage over the tanh baseline, so no practical win was observed.", "metrics": { "baseline": "tanh RNN: test loss 0.69302, accuracy 0.5121, first-token input gradient 1.26e-10, final hidden norm 0.523", "idea": "dual-unitary: test loss 0.69327, accuracy 0.5000, first-token input gradient 8.91e-4, final hidden norm 3.201; homogeneous transition norm preservation error <=4.44e-15" }, "how_to_run": "CUDA_VISIBLE_DEVICES='' /home/maxwelhelp/main/bin/python3 dual_unitary_experiment.py", "files": [ "dual_unitary_experiment.py", "results.json" ], "limitations": "Only a small fixed-orthogonal-transition delayed-bit task was tested, not sequential or permuted MNIST. The auxiliary/unitary parameters were not learned, parameter counts were not tightly matched, and the additive input drive means the full forced recurrence does not preserve state norm exactly—only the recurrent transition does." }