# Convergent Pearson-Correlation Recurrent Layer

- ID: 1748
- Canonical URL: https://synthcore.org/idea/1748/convergent-pearson-correlation-recurrent-layer
- API JSON: https://synthcore.org/api/idea/1748.json
- API Markdown: https://synthcore.org/api/idea/1748.md
- Verification status: unverified
- Source: [arXiv:2608.03404](https://arxiv.org/abs/2608.03404)
- Category: dynamics
- Solves: stability, sample-efficiency
- ML areas: optimizer, training-dynamics, embedding, rnn
- Math tags: dynamical-systems, linear-algebra, geometry, bifurcations
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Insert a recurrent layer that repeatedly replaces a three-by-three feature affinity matrix by the Pearson correlations of its rows. Unlike an unconstrained recurrent affinity update, the state remains a valid correlation matrix, becomes rank at most two after one step, and in dimension three converges globally to one of seven fixed points. Use the converged patterned fixed point as a differentiable or stop-gradient clustering/relational embedding, while monitoring rank and kernel-coordinate contraction as a principled stopping rule.

## Mathematical statement

For a matrix A with nonconstant rows, let C(A) be the matrix with entries C(A)_{ij}=\operatorname{corr}(a_i,a_j), let \mathbf{1} be the all-ones vector, and let H_n=I_n-\mathbf{1}\mathbf{1}^{T}/n. The row-wise Gram factorization gives \operatorname{rank}C(A)=\operatorname{rank}(AH_n), so any n-by-n correlation state reaches rank at most n-1 after one iteration. For n=3 and a rank-two state P=XX^T, choose a nonzero q spanning \ker P=\ker X^T, and let D=\operatorname{diag}(d_1,d_2,d_3), where d_i is the centered Euclidean norm of row i of X. The next state has kernel spanned by Dq. When q_1+q_2+q_3
eq0, the exact relation d_i^2-d_j^2=[2\det(X^TH_3X)/(q_1+q_2+q_3)](q_i-q_j) supplies monotone projective-ratio coordinates. The paper proves that every admissible 3-by-3 orbit converges to one of seven fixed points; three patterned rank-one points are locally asymptotically stable, while four other points are unstable and have exceptional, measure-zero basins.

## Key formulas

- $$C(A)_{ij}=\frac{(a_i-\bar a_i\mathbf{1})^{T}(a_j-\bar a_j\mathbf{1})}{\|a_i-\bar a_i\mathbf{1}\|_2\,\|a_j-\bar a_j\mathbf{1}\|_2},\qquad H_n=I_n-\frac{1}{n}\mathbf{1}\mathbf{1}^{T}.$$
- $$\operatorname{rank}C(A)=\operatorname{rank}(AH_n)\leq n-1,\qquad P_{t+1}=C(P_t).$$
- $$P=XX^{T},\quad q\in\ker P\setminus\{0\},\quad \ker C(P)=\operatorname{span}\{Dq\},\quad D=\operatorname{diag}(d_1,d_2,d_3).$$
- $$d_i^2-d_j^2=\frac{2\det(X^{T}H_3X)}{q_1+q_2+q_3}(q_i-q_j),\qquad q_1+q_2+q_3\neq0.$$

## Implementation notes

1. Integration point: for each training example, take three learned feature vectors z_1,z_2,z_3 from an MLP, CNN, transformer token group, or graph-neighborhood sampler. Stack them as rows of Z and define P_0=C(Z); then apply K recurrent updates P_{t+1}=C(P_t), with an epsilon-stabilized Pearson denominator. Feed P_K, its leading eigenvector, or the sign pattern of its off-diagonal entries to a downstream classifier or contrastive loss. Keep P symmetric by replacing it with (P+P^T)/2 and set the diagonal to one after each update. 2. Pseudocode: `P = pearson_rows(Z); for t in range(K): P = pearson_rows(P); P = symmetrize_and_unit_diagonal(P); loss = task_loss(head(P), y)`. Estimate the kernel vector q as the eigenvector of P with smallest eigenvalue, compute a projective ratio such as r_t=max_{i,j}|q_i/q_j| in a fixed sign chart, and stop when both `||P_{t+1}-P_t||_F` and the change in r_t are below tolerance. 3. Computed from the paper: the rank bound, elliptope invariance, kernel update, and the prediction that only patterned fixed points are attracting. Estimated empirically: finite-precision Jacobian eigenvalues, convergence rate, and which basin a learned Z enters. Use a straight-through estimator or unroll K steps; detach q and stopping decisions if gradients become unstable. 4. First cheap experiment: train a three-token MLP or one-layer transformer on synthetic data where each example contains one of three planted sign-pattern clusters, comparing a standard learned 3-by-3 affinity matrix against this recurrent correlation layer and a single Pearson layer. Sweep random initializations and K. The falsifiable signature is a sharp collapse of the smallest eigenvalue toward zero after the first update, followed by convergence of P_t to one of seven numerically identifiable fixed points; for generic initial states, the fraction not converged by K should decay rapidly, while points initialized near the four unstable fixed points should form a vanishingly small exceptional set. Verify the predicted rank bound to numerical tolerance and compare measured fixed-point attraction against finite-difference Jacobian eigenvalues.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.