# Carrier-Probed Hidden-State Training

- ID: 1351
- Canonical URL: https://synthcore.org/idea/1351/carrier-probed-hidden-state-training
- API JSON: https://synthcore.org/api/idea/1351.json
- API Markdown: https://synthcore.org/api/idea/1351.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.24134](https://arxiv.org/abs/2607.24134)
- Category: dynamics
- Solves: accuracy, sample-efficiency, stability
- ML areas: ssm, rnn, world-model, data-augmentation, training-dynamics
- Math tags: control-theory, dynamical-systems, linear-algebra, optimization
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

When a neural state-space model has latent directions that are invisible under normal inputs, add a small structured carrier to the input or hidden-state update during selected training windows. The carrier changes local measurement and transition projections, analogous to the paper's carrier-dependent measurement and force projections, and can reveal modes that passive training leaves unconstrained.

## Mathematical statement

Let the passive model be $z_{t+1}=f_\theta(z_t,u_t)$ and $y_t=h_\theta(z_t)$. Introduce a known carrier $m_t$ with amplitude $a$ and define $u_t^{(m)}=u_t+a m_t$. Around the carrier-driven trajectory, define $A_t(m)=\partial f_\theta/\partial z$, $B_t(m)=\partial f_\theta/\partial u$, and $C_t(m)=\partial h_\theta/\partial z$. The carrier is useful when it increases the smallest observability or reachability eigenvalue. Select it by maximizing a log-determinant information score penalized by carrier energy, where $\delta>0$ stabilizes singular matrices and $\alpha,\beta\ge0$ control reachability and excitation cost.

## Key formulas

- $$u_t^{(m)}=u_t+a\,m_t,\qquad m_t\in\mathbb{R}^{m},\quad \|m_t\|_2\le 1,$$
- $$W_o(m)=\sum_{t=0}^{H-1}\Phi_t(m)^T C_t(m)^TC_t(m)\Phi_t(m)\,\Delta t,$$
- $$W_r(m)=\sum_{\tau=0}^{H-1}\Psi_{\tau+1}(m)B_\tau(m)B_\tau(m)^T\Psi_{\tau+1}(m)^T\,\Delta t,$$
- $$m^*=\arg\max_m\left[\log\det(W_o(m)+\delta I)+\alpha\log\det(W_r(m)+\delta I)-\beta a^2\right].$$

## Implementation notes

Apply this to an SSM, GRU, or world model during training rather than necessarily at deployment. Maintain a small candidate library consisting of sinusoidal carriers at frequencies $\omega_k$, binary pseudorandom sequences, channel-wise sign patterns, and short orthogonal pulses. For each minibatch, run the model passively and with $u_t+a m_t$. Compute Jacobians $A_t,B_t,C_t$ using automatic differentiation or Jacobian-vector products, propagate finite-horizon transition matrices, and evaluate the log-determinant score. Select the highest-scoring carrier, train on that rollout, and add an output-consistency loss requiring the model to predict the carrier-induced change. Pseudocode is: sample candidates $m$; roll out $z^{(m)}$; estimate $A_t(m),B_t(m),C_t(m)$; accumulate $W_o(m)$ and $W_r(m)$; score each candidate; optimize $\mathcal{L}_{task}+\gamma\|\Delta y_{pred}-\Delta y_{actual}\|^2$ for the selected carrier. The paper supplies the active-projection and Gramian mechanism; candidate search and Jacobian estimation are empirical neural adaptations. Begin with a GRU on a synthetic system containing one observable fast mode and one passive-hidden slow mode, then test a neural ODE on Lorenz-63 with one latent coordinate masked from the output. The predicted signature is a carrier threshold: below amplitude $a_c$, $\lambda_{\min}(W_o)$ remains near zero; above it, the eigenvalue should increase approximately as $a^2$ for small perturbations because the carrier-induced output projection is first order in $a$ and the Gramian is quadratic. Test whether hidden-coordinate recovery begins at this transition.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a two-state nonlinear SSM with an observed fast mode and a passive-hidden slow mode, plus finite-horizon observability/reachability Gramian computation and carrier candidate scoring. The passive hidden Gramian eigenvalue was exactly 0, and the carrier-induced minimum observability eigenvalue followed a^1.999 versus the predicted a^2, with coefficient variation 0.095%. Under identical noise, passive q recovery had absolute error 0.8, while a=0.16 probing reduced it to 0.009; candidate scoring selected the carrier with the strongest hidden observability. The mechanism manifested in this toy setting, but this is not evidence yet for GRUs, neural ODEs, or general learned models.

### Mechanism check

- Verdict: Built a two-state nonlinear SSM with an observed fast mode and a passive-hidden slow mode, plus finite-horizon observability/reachability Gramian computation and carrier candidate scoring. The passive hidden Gramian eigenvalue was exactly 0, and the carrier-induced minimum observability eigenvalue followed a^1.999 versus the predicted a^2, with coefficient variation 0.095%. Under identical noise, passive q recovery had absolute error 0.8, while a=0.16 probing reduced it to 0.009; candidate scoring selected the carrier with the strongest hidden observability. The mechanism manifested in this toy setting, but this is not evidence yet for GRUs, neural ODEs, or general learned models.
- Confidence: 8/10
- Limitations: Only a hand-designed two-dimensional Euler SSM was tested; no GRU/neural ODE, minibatch training, output-consistency loss, automatic Jacobian implementation, FLOP/runtime comparison, or deployment impact was evaluated. The threshold rule is an arbitrary numerical criterion, and very small-amplitude recovery is unstable under noise despite the clean Gramian scaling.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.45895
- Paired wins: 6/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/530/bench_report.json)
- [candidate_check.py](https://synthcore.org/code/530/candidate_check.py)
- [candidate_results.json](https://synthcore.org/code/530/candidate_results.json)
- [carrier_bench.py](https://synthcore.org/code/530/carrier_bench.py)
- [carrier_probe.py](https://synthcore.org/code/530/carrier_probe.py)
- [report.md](https://synthcore.org/code/530/report.md)
- [report_bench_2026-08-31T170154.md](https://synthcore.org/code/530/report_bench_2026-08-31T170154.md)
- [results.json](https://synthcore.org/code/530/results.json)
- [Download all files as ZIP](https://synthcore.org/download/530)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.