Minor-preserving complex network layer
Implementation & benchmark of arXiv:2608.27939 — Entropy and domination for quasi-Hitchin representations
Source paper: Entropy and domination for quasi-Hitchin representations arXiv:2608.27939 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Parameterize a complex linear layer as a product of sparse triangular network factors whose positive modulus version is totally nonnegative. The layer can use phase cancellation for expressive transformations, while selected minors remain bounded by explicitly computable positive minors, giving a structured alternative to unconstrained dense complex weights.
Formulas
Mathematical statement
The paper uses weighted planar networks, where matrix entries and minors are path sums. For a complex matrix $A$, let $A_0$ be the same network with every edge weight replaced by its modulus. The key minor domination statement is $|\Delta_{I,J}(A)|\leq\Delta_{I,J}(A_0)$, with strict inequality for suitable $I,J$ when at least one parameter has a non-real phase. Here $I,J\subset\{1,\ldots,n\}$ are equally sized source and sink index sets, $\Delta_{I,J}(A)$ is the determinant of the submatrix with rows $I$ and columns $J$, and $1\leq |I|=|J|<n$. A path family contributes signed or complex terms to $\Delta_{I,J}(A)$, whereas the positive companion sums their magnitudes. The transferable property is that phase-induced cancellation can be measured relative to a positive structural envelope, allowing the model to retain complex expressivity while controlling rank collapse and amplification.
Implementation notes
Replace a dense complex projection or feed-forward matrix with a product of $L$ sparse factors. Let each $T_k$ be unit-diagonal lower or upper triangular with nonnegative off-diagonal magnitudes, let each $E_k$ be positive diagonal, and attach independent phases $e^{i\phi}$ to the trainable edges: $w_e=r_e e^{i\phi_e}$ with $r_e\geq0$. The positive companion uses the same magnitudes and sets all phases to zero. In the forward pass compute z=x; z0=abs(x); for k in 1..L: z=Tk_complex(Ek_complex(z)); z0=abs(Tk_complex)@(abs(Ek_complex)@z0). On each batch, sample two to eight row-column pairs $(I,J)$ with sizes $r=1$ and $r=2$, compute the corresponding determinants using torch.linalg.det, and add $\lambda_{minor}\mathcal{L}_{minor}$ only when the complex minor is below a chosen fraction of its positive companion. For $r=1$, this is an entrywise constraint; for $r=2$, it detects pairwise rank collapse. The paper-derived quantities are the path-family determinant expansion and the inequality $|\Delta_{I,J}(A)|\leq\Delta_{I,J}(A_0)$; subset sampling, margins, and penalty weights are empirical. First test on a 4-layer complex MLP or complex attention value projection on CIFAR-10, comparing dense complex, real dense, and the structured factorization at matched parameter count. Track accuracy, Jacobian condition numbers, gradient variance, and throughput. Success means equal or better accuracy with fewer near-singular Jacobians and stable training at larger learning rates.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.