# Finite-Support Sparse Correction Horizon

- ID: 2457
- Canonical URL: https://synthcore.org/idea/2457/finite-support-sparse-correction-horizon
- API JSON: https://synthcore.org/api/idea/2457.json
- API Markdown: https://synthcore.org/api/idea/2457.md
- Verification status: unverified
- Source: [arXiv:2608.17464](https://arxiv.org/abs/2608.17464)
- Category: dynamics
- Solves: stability, speedup, memory
- ML areas: optimizer, rnn, ssm, training-dynamics
- Math tags: control-theory, dynamical-systems, spectral-theory, optimization
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Represent an iterative neural computation as a controlled dynamical system and learn sparse residual corrections that are active only for a finite prefix of iterations. Estimate local stable and anti-stable subspaces of the hidden-state Jacobian, increase the correction horizon only while the anti-stable component exceeds a tolerance, and force later controls to zero. This produces adaptive-depth inference with a quantitative stopping criterion.

## Mathematical statement

The paper studies the discrete system x_{k+1}=A x_k+B u_k with weighted sparse cost J(u)=sum_{k=0}^infinity alpha^k ||u_k||_1 and terminal condition alpha^k x_k -> 0. The assumption is that alpha A has no eigenvalues on the unit circle. Decompose A=T diag(A_s,A_u) T^{-1}, where rho(alpha A_s)<1 and rho(alpha A_u)>1. Once the anti-stable state x_u=T_u^{-1}x is driven sufficiently close to zero, setting u_k=0 leaves only a stable tail satisfying the weighted decay condition. For a neural transition h_{k+1}=F_theta(h_k)+G_phi(h_k)c_k, estimate the local Jacobian J_k=dF_theta/dh and its stable and anti-stable projectors P_s and P_u. Stop applying corrections when ||P_u h_k||_2 is below epsilon and the stable tail contracts. The predicted active horizon is the first k satisfying this condition.

## Key formulas

- $$x_{k+1}=Ax_k+Bu_k,\qquad J(u)=\sum_{k=0}^{\infty}\alpha^k\lVert u_k\rVert_1,\qquad \alpha^k x_k\to0.$$
- $$A=T\begin{pmatrix}A_s&0\\0&A_u\end{pmatrix}T^{-1},\qquad \rho(\alpha A_s)<1<\rho(\alpha A_u).$$
- $$h_{k+1}=F_\theta(h_k)+G_\phi(h_k)c_k,\qquad \mathcal{L}=\mathcal{L}_{\mathrm{task}}+\lambda\sum_{k=0}^{N-1}\alpha^k\lVert c_k\rVert_1.$$
- $$N=\min\left\{n:\ \lVert\widehat P_u h_n\rVert_2\le\varepsilon\ \text{and}\ \rho(\alpha\widehat J_s)^j\lVert\widehat P_s h_n\rVert_2\le\varepsilon\ \text{for all }j\ge0\right\}.$$

## Implementation notes

1. Integration point: place the mechanism inside an unrolled fixed-point solver, recurrent network, learned optimizer, or iterative denoising module. Use a base transition h_next=F_theta(h) and a controller c=C_phi(h) that supplies a residual through G_phi(h)c. Train with the task loss plus the weighted L1 penalty in the formulas. At inference, apply corrections only while the anti-stable state is above tolerance. 2. Pseudocode: initialize h=h0. For k from 0 to N_max minus 1, estimate a few dominant eigenvalues or singular directions of the local Jacobian using power iteration and Jacobian-vector products. Construct approximate projectors P_u and P_s from eigenvectors whose magnitudes are respectively above and below 1 divided by alpha. If norm(P_u h) is greater than epsilon, compute c=C_phi(h), update h=F_theta(h)+G_phi(h)c, and record alpha^k times the L1 norm of c. Otherwise set c=0, update h=F_theta(h), and stop after one or two contraction checks. 3. The paper supplies the sparse objective, spectral decomposition, finite-support argument, and weighted decay condition. The neural implementation estimates the nonlinear Jacobian, projectors, and tolerance empirically; recompute them every M steps to reduce cost. 4. First cheap experiment: compare a 20-step GRU refinement model against dense corrections and a fixed-depth sparse-gated baseline on MNIST or CIFAR-10. Sweep lambda, alpha, and epsilon. The falsifiable prediction is a sharp increase in the average active correction horizon when rho(alpha J_u) crosses 1. After stopping, hidden-state perturbations should decay geometrically with slope approximately log rho(alpha J_s), and the measured stopping time should be the first step where norm(P_u h) is below epsilon. Test stability for ten additional correction-free steps.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.