Convergent Pearson-Correlation Recurrent Layer
Implementation & benchmark of arXiv:2608.03404 — Fixed Points, Stability, Basin Geometry, and Global Convergence of the $3\times3$ Correlation Map
Source paper: Fixed Points, Stability, Basin Geometry, and Global Convergence of the $3\times3$ Correlation Map arXiv:2608.03404 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Insert a recurrent layer that repeatedly replaces a three-by-three feature affinity matrix by the Pearson correlations of its rows. Unlike an unconstrained recurrent affinity update, the state remains a valid correlation matrix, becomes rank at most two after one step, and in dimension three converges globally to one of seven fixed points. Use the converged patterned fixed point as a differentiable or stop-gradient clustering/relational embedding, while monitoring rank and kernel-coordinate contraction as a principled stopping rule.
Formulas
Mathematical statement
For a matrix A with nonconstant rows, let C(A) be the matrix with entries C(A)_{ij}=\operatorname{corr}(a_i,a_j), let \mathbf{1} be the all-ones vector, and let H_n=I_n-\mathbf{1}\mathbf{1}^{T}/n. The row-wise Gram factorization gives \operatorname{rank}C(A)=\operatorname{rank}(AH_n), so any n-by-n correlation state reaches rank at most n-1 after one iteration. For n=3 and a rank-two state P=XX^T, choose a nonzero q spanning \ker P=\ker X^T, and let D=\operatorname{diag}(d_1,d_2,d_3), where d_i is the centered Euclidean norm of row i of X. The next state has kernel spanned by Dq. When q_1+q_2+q_3
eq0, the exact relation d_i^2-d_j^2=[2\det(X^TH_3X)/(q_1+q_2+q_3)](q_i-q_j) supplies monotone projective-ratio coordinates. The paper proves that every admissible 3-by-3 orbit converges to one of seven fixed points; three patterned rank-one points are locally asymptotically stable, while four other points are unstable and have exceptional, measure-zero basins.
Implementation notes
1. Integration point: for each training example, take three learned feature vectors z_1,z_2,z_3 from an MLP, CNN, transformer token group, or graph-neighborhood sampler. Stack them as rows of Z and define P_0=C(Z); then apply K recurrent updates P_{t+1}=C(P_t), with an epsilon-stabilized Pearson denominator. Feed P_K, its leading eigenvector, or the sign pattern of its off-diagonal entries to a downstream classifier or contrastive loss. Keep P symmetric by replacing it with (P+P^T)/2 and set the diagonal to one after each update. 2. Pseudocode: P = pearson_rows(Z); for t in range(K): P = pearson_rows(P); P = symmetrize_and_unit_diagonal(P); loss = task_loss(head(P), y). Estimate the kernel vector q as the eigenvector of P with smallest eigenvalue, compute a projective ratio such as r_t=max_{i,j}|q_i/q_j| in a fixed sign chart, and stop when both ||P_{t+1}-P_t||_F and the change in r_t are below tolerance. 3. Computed from the paper: the rank bound, elliptope invariance, kernel update, and the prediction that only patterned fixed points are attracting. Estimated empirically: finite-precision Jacobian eigenvalues, convergence rate, and which basin a learned Z enters. Use a straight-through estimator or unroll K steps; detach q and stopping decisions if gradients become unstable. 4. First cheap experiment: train a three-token MLP or one-layer transformer on synthetic data where each example contains one of three planted sign-pattern clusters, comparing a standard learned 3-by-3 affinity matrix against this recurrent correlation layer and a single Pearson layer. Sweep random initializations and K. The falsifiable signature is a sharp collapse of the smallest eigenvalue toward zero after the first update, followed by convergence of P_t to one of seven numerically identifiable fixed points; for generic initial states, the fraction not converged by K should decay rapidly, while points initialized near the four unstable fixed points should form a vanishingly small exceptional set. Verify the predicted rank bound to numerical tolerance and compare measured fixed-point attraction against finite-difference Jacobian eigenvalues.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.