# Star-Symmetric Delayed State Layer

- ID: 3084
- Canonical URL: https://synthcore.org/idea/3084/star-symmetric-delayed-state-layer
- API JSON: https://synthcore.org/api/idea/3084.json
- API Markdown: https://synthcore.org/api/idea/3084.md
- Verification status: unverified
- Source: [arXiv:2609.02801](https://arxiv.org/abs/2609.02801)
- Category: architecture
- Solves: stability, accuracy
- ML areas: ssm, rnn, graph-nn
- Math tags: spectral-theory, linear-algebra, representation-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace an untied stack of recurrent or polynomial-filter blocks with a delayed recurrence whose coefficients repeat with period r and whose characteristic polynomial has exact (r+1)-fold rotational symmetry. The resulting layer couples the current state to a state r steps back, producing a structured spectrum rather than arbitrary eigenvalues and potentially improving long-horizon propagation with fewer parameters.

## Mathematical statement

The paper defines monic polynomials by P_(n+1)(z) = z P_n(z) - gamma_(n-r) P_(n-r)(z) for n >= r, with P_0 = 1 and P_(-1) = ... = P_(-r) = 0. Here r >= 1 is the delay order and every gamma_j > 0 is a recurrence coefficient. The identity P_n(omega_(r+1) z) = omega_(r+1)^n P_n(z), where omega_(r+1) = exp(2 pi i/(r+1)), implies that zeros occur on the star Sigma = union from k=0 to r of {x omega_(r+1)^k : x >= 0}. Transfer this recurrence to vector states by replacing the scalar variable z with a learned linear operator A and defining h_(t+1) = A h_t - gamma_(t-r) h_(t-r). The state-transition characteristic polynomial is then governed by the same delayed polynomial structure. Periodic coefficients can be imposed with gamma_j = alpha_(j mod r), while positivity is enforced by a softplus parameterization. For real networks, use real matrices directly, or represent complex rotations by 2-by-2 real blocks.

## Key formulas

- $$P_{n+1}(z)=zP_n(z)-\gamma_{n-r}P_{n-r}(z),\qquad n\ge r$$
- $$P_0=1,\qquad P_{-1}=\cdots=P_{-r}=0$$
- $$P_n(\omega_{r+1}z)=\omega_{r+1}^{n}P_n(z),\qquad \omega_{r+1}=e^{\frac{2\pi i}{r+1}}$$
- $$\Sigma=\bigcup_{k=0}^{r}\{x\omega_{r+1}^{k}:x\ge0\}$$

## Implementation notes

Integrate this at the sequence-model state update or as a graph polynomial-filter block. Let h_t in R^d be the hidden state, let A in R^(d by d) be a learned linear operator, and choose delay r in {1, 2, 3}. Maintain a queue containing h_t, h_(t-1), ..., h_(t-r). Use positive coefficients by parameterizing gamma_j = softplus(u_j) + epsilon, with either one coefficient per phase j modulo r or a slowly varying schedule. The update is: (1) compute v = A h_t + B x_(t+1); (2) retrieve q = h_(t-r); (3) set h_(t+1) = v - gamma_(t-r) q; (4) apply normalization and the task nonlinearity only after this structured recurrence. In pseudocode: `v = A @ h[t] + B @ x[t+1]`; `g = softplus(u[t % r]) + eps`; `h[t+1] = norm(v - g * h[t-r])`. The exact mathematical structure supplied by the paper is the delayed polynomial and rotational symmetry; stability and optimization benefits must be estimated empirically because the extracted paper material does not provide a neural convergence theorem. Initialize A with spectral radius near 1 and initialize gamma values small enough that the augmented delay-state matrix has spectral radius near or below 1. First test on copy, adding, and sequential-MNIST tasks using a two-layer structured SSM with width 128. Compare against an untied r-lag residual recurrence with the same parameter count and FLOPs. Track validation loss versus sequence length, gradient norm variance, hidden-state norm, and failure rate across random seeds. Success means slower accuracy degradation on long sequences, fewer exploding or vanishing runs, or faster loss descent without increasing parameter count.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.