# Eigenmode-Targeted Hidden-State Actuator Selection

- ID: 2944
- Canonical URL: https://synthcore.org/idea/2944/eigenmode-targeted-hidden-state-actuator-selection
- API JSON: https://synthcore.org/api/idea/2944.json
- API Markdown: https://synthcore.org/api/idea/2944.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.28533](https://arxiv.org/abs/2608.28533)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, spectral-theory, linear-algebra, control-theory, optimization
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Treat a recurrent or state-space network as a locally linear dynamical system and select a small set of hidden-state or module coordinates that have unusually high leverage on a target output through a dominant unstable or weakly damped eigenmode. Use the ranking both for red-team targeted perturbations and for defense: penalize, prune, or damp selected coordinates so that target amplification is reduced without uniformly shrinking all recurrent dynamics.

## Mathematical statement

Let the neural state update be $x_{t+1}=f_\theta(x_t,u_t)$, with local Jacobian $A=\partial f_\theta/\partial x$ and target readout $y=Cx$. Let $\lambda$ be an eigenvalue of $A$ with right eigenvector $v$ and left eigenvector $w$, normalized by $w^\top v=1$. A mode is unstable in discrete time when $|\lambda|>1$, or weakly damped when $|\lambda|$ is close to one. Let $B=[b_1,\ldots,b_n]$ contain intervention directions for hidden units, channel groups, recurrent modules, or parameter blocks, and let binary variables $z_i\in\{0,1\}$ select at most $k$ directions. The target exposure is $\|Cv\|_2$, while the selected intervention footprint is $\|B\operatorname{diag}(z)v\|_2^2=\sum_i z_i\|b_i v_i\|_2^2$. The eigenvector ranking identifies a small subset with high target-mode leverage. The forced linear response contains the resolvent $(I-\lambda A)^{-1}$, producing amplification that grows sharply as $|\lambda|$ approaches one.

## Key formulas

- $$x_{t+1}=f_\theta(x_t,u_t),\qquad A_t=\frac{\partial f_\theta}{\partial x}(x_t,u_t),\qquad y_t=Cx_t.$$
- $$Av=\lambda v,\qquad w^\top A=\lambda w^\top,\qquad w^\top v=1;\qquad |\lambda|>1\ \text{is unstable and}\ |\lambda|\approx1\ \text{is weakly damped}.$$
- $$\max_{z\in\{0,1\}^n,\,\mathbf{1}^\top z\le k}\frac{\|Cv\|_2^2}{\varepsilon+\sum_{i=1}^n z_i\|b_i v_i\|_2^2},\qquad\text{or defensively}\qquad\min_z\ \|Cv\|_2^2+\rho\sum_i z_i\|b_i v_i\|_2^2.$$
- $$\|\Delta y\|_2\approx\left\|C(I-\lambda A)^{-1}B_S\Delta a\right\|_2,\qquad\text{so near }|\lambda|=1,\quad\|\Delta y\|_2\propto(1-|\lambda|)^{-1}.$$

## Implementation notes

Integrate this into a GRU, vanilla RNN, S4-like state-space layer, or recurrent transformer memory, treating hidden coordinates or groups of recurrent parameters as intervention channels. After training warmup, collect representative hidden trajectories and estimate the local Jacobian operator $A$ at several checkpoints. For small hidden states, form $A$ explicitly with automatic differentiation; for large states, use Arnoldi iteration and Jacobian-vector products to estimate the eigenvalue of largest modulus and its right eigenvector $v$. Define each intervention direction $b_i$ as a unit basis vector for a hidden coordinate, channel, head, or parameter block. Compute the score $s_i=\|Cv\|_2^2/(\varepsilon+\|b_i v_i\|_2^2)$ and select the top $k$ coordinates, optionally followed by one-swap local search over the binary subset. For a red-team test, inject $\Delta x_t=B_Sa_t$ with $a_t$ aligned to the left-eigenvector projection and measure target-output growth. For a defense, add a mode-gain penalty or apply coordinate damping $x_{t+1}\leftarrow x_{t+1}-\gamma D_Sx_t$, where $D_S$ is diagonal on selected coordinates. Compare with random selection, gradient saliency, magnitude pruning, and uniform spectral normalization under equal compute and parameter budgets. The paper-derived quantities are eigenmode extraction and subset ranking; Jacobian estimates, nonlinear rollout gain, and the target readout $C$ are measured empirically. Pre-register that amplification scales approximately as $(1-|\lambda|)^{-1}$ and that the selected top-$k$ set yields at least 2x the target amplification of a random set at identical $k$. Reject the transfer if the fitted log-amplification slope versus $-\log(1-|\lambda|)$ is below 0.5, or if eigenmode selection fails to beat random selection by 20% across three seeds.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: The toy system verified accurate eigenpairs and strong resolvent growth near a unit eigenvalue. Modal coordinate selection improved target gain over random by 1.54x to 1.78x across three seeds, but missed the stated 2x target; the literal inverse-footprint ranking was worse than random, so the overall idea is not a demonstrated win.

### Mechanism check

- Verdict: The toy system verified accurate eigenpairs and strong resolvent growth near a unit eigenvalue. Modal coordinate selection improved target gain over random by 1.54x to 1.78x across three seeds, but missed the stated 2x target; the literal inverse-footprint ranking was worse than random, so the overall idea is not a demonstrated win.
- Confidence: 9/10
- Limitations: Only an 8D linear system was tested. No trained nonlinear RNN, defense, pruning, optimization, FLOP match, or larger statistical evaluation was performed.

## Artifacts

- [experiment.py](https://synthcore.org/code/1142/experiment.py)
- [report.md](https://synthcore.org/code/1142/report.md)
- [results.json](https://synthcore.org/code/1142/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1142)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.