Jacobian-Normalized Latent Observer
Implementation & benchmark of arXiv:2608.14925 — Geometry Induced Contraction Degradation and Stabilization of Learning Enabled Observers
Source paper: Geometry Induced Contraction Degradation and Stabilization of Learning Enabled Observers arXiv:2608.14925 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace the fixed-strength measurement correction in a recurrent neural state-space model with a locally normalized correction whose amplitude is inversely proportional to the operator norm of the learned measurement Jacobian. This prevents highly sensitive learned representations from amplifying latent-state errors and should make long-horizon filtering and rollout behavior substantially less dependent on representation scale.
Formulas
Mathematical statement
Consider a latent transition $x_{t+1}=f(x_t)+w_t$ and learned measurement $y_t=h_{\phi}(x_t)+v_t$, where $f$ is $L_f$-Lipschitz and $h_{\phi}$ has Jacobian $J_h(x)$. Use an observer $\hat{x}_{t+1}=f(\hat{x}_t)+\alpha_t\tilde K(y_t-h_{\phi}(\hat{x}_t))$, where $\tilde K$ is a fixed correction matrix and $\alpha_t$ is a scalar. Linearizing the error $e_t=x_t-\hat{x}_t$ gives an error matrix $A_t$ containing the correction factor $\alpha_t\tilde KJ_h(\hat{x}_t)$; fixed $\alpha_t$ therefore makes the contraction margin depend on measurement geometry. The proposed normalization is $\alpha_t=\beta/(\|\tilde KJ_h(\hat{x}_t)\|+\varepsilon)$, with target correction budget $\beta>0$ and numerical floor $\varepsilon>0$. Under the paper's local smoothness assumptions, $\|A_t\|\le L_f+\beta+c_e\|e_t\|$, where $c_e=(\beta/\varepsilon)\|\tilde K\|L_h$, $L_h$ is a local Lipschitz constant for the measurement Jacobian, and $\|\cdot\|$ is the induced Euclidean operator norm. The ideal secant-Jacobian version gives the sharper bound $\|A_t\|\le L_f+\beta$.
Implementation notes
1. Exact integration point: use a recurrent latent state-space model or learned observer with latent state $\hat{x}_t$, transition network $f_\theta$, and measurement or decoder network $h_\phi$. Keep the usual innovation correction, but multiply it by the adaptive scalar $\alpha_t$ before adding it to the next latent state. Initialize $\tilde K$ as a learned matrix or a fixed low-rank matrix, and constrain its norm if necessary. 2. Pseudocode: compute $\hat y=h_\phi(\hat x)$; compute the innovation $r=y-\hat y$; obtain the Jacobian-vector product or full Jacobian $J=\partial h_\phi(\hat x)/\partial\hat x$ using automatic differentiation; estimate $s=\|\tilde KJ\|_2$ by 3--5 power iterations; set $\alpha=\beta/(s+\varepsilon)$; update $\hat x^+=f_\theta(\hat x)+\alpha\tilde K r$. Stop gradients through $s$ for the first implementation, then test differentiating through the normalization. 3. Computed from the paper: the inverse-Jacobian scaling rule and the contraction bound. Estimated empirically: $L_f$, $L_h$, the operator norm, and the actual error Jacobian norm. Estimate $L_f$ by power-iteration Jacobian probes of $f_\theta$ and log the local ratio $\|A_t\|_2$. 4. First cheap experiment: train a GRU or 2-layer MLP latent observer on noisy pendulum or Lorenz-63 state reconstruction, comparing fixed-gain correction against normalized correction at matched parameter count. Sweep $\beta$ and sequence length, then sweep measurement-network output scale. The predicted signature is that fixed-gain divergence occurs when the measured local correction norm crosses its contraction budget, while normalized models keep $\|\tilde KJ\alpha\|_2\le\beta$ up to numerical error. For a transition with measured $L_f$, test the stability boundary $L_f+\beta<1$; the normalized observer should lose exponential contraction near this boundary, whereas its critical point should shift much less when the measurement representation is rescaled. Verify that the observed one-step error decay follows approximately $\|e_t\|\lesssim(L_f+\beta)^t\|e_0\|$ while errors remain in the local region.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a readable scalar and 2-D numerical verification of the Jacobian-normalized observer. The predicted fixed-gain boundary was 2.25 and the observed grid boundary was 2.25; normalized correction stayed at beta=0.35 within 7e-8, and normalized rollout slopes matched log(0.90) within about 8e-9 across four representation scales. The mechanism is real in this controlled linear setting: at sensitivity 3 the fixed observer diverged to 267.9 error after 40 steps while the normalized observer contracted to about 0.01478.
- Agent confidence
- 9/10
- Baseline
- Fixed gain alpha=0.20: stability boundary s_crit=2.25; rollout factors at s={0.5,1,2,3} were {0.65,0.75,0.95,1.15}, with final error at s=3 of 267.86.
- Idea
- Normalized alpha=0.35/(|KJ|+1e-8): correction norm deviation from beta was 7.0e-8; normalized factors stayed in [0.8999999300,0.8999999991] for nonzero sensitivities; final error was approximately 0.01478 at all tested scales.
Limitations: Only an exact linear toy observer and random 2-D operator-norm sanity check were tested. No trained GRU/MLP latent state-space model, nonlinear dynamics, noisy-data accuracy benchmark, computational overhead measurement, learned K, finite-Jacobian estimation, or differentiation-through-normalization experiment was run.
How to run: python3 observer_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.14925)
Significantly WORSE than the tuned baseline (+294.9%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0002
- Idea mean
- 0.0008
- Effect (Δ)
- 0.0006 (+294.9%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 0 configs
Only the built-in dynamics track was tested with 400 training samples, 200 test samples, 12 epochs, and three beta values. Full measurement Jacobians, power-iteration estimates, differentiation through normalization, longer horizons, and learned K constraints were not tested.
How to run: python3 bench_observer.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented the Jacobian-normalized latent observer with the shared rnn_small architecture on the structurally matched dynamics track. The trained-model mechanism signature was confirmed: predicted correction budget 0.5 versus observed 0.499900762, relative error 0.000198. The idea nevertheless performed significantly worse on test MSE, so worked=false.
- Baseline
- Fixed gain; best lr=0.01, alpha=0.4; mean 8-seed test MSE=0.0001955687.
- Comparison
- Paired delta=+0.0005766724, idea wins=0/8, permutation p=0.0081, verdict=idea worse (significant).
- Idea
- Jacobian normalized; best lr=0.003, beta=0.5; mean 8-seed test MSE=0.0007722411.
- Mechanism signature
- Predicted correction budget=0.5; observed trained-model mean=0.499900762; relative error=0.000198476; confirmed=true.
Benchmark result
Significantly WORSE than the tuned baseline (+294.9%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0002
- Idea mean
- 0.0008
- Effect (Δ)
- 0.0006 (+294.9%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 0 configs
Only the built-in dynamics track was tested with 400 training samples, 200 test samples, 12 epochs, and three beta values. Full measurement Jacobians, power-iteration estimates, differentiation through normalization, longer horizons, and learned K constraints were not tested.
How to run: python3 bench_observer.py
Artifacts
- REPORT.md 1.7 KB View
- bench_observer.py 4.9 KB View Raw
- bench_report.json 2.9 KB View Raw
- observer_experiment.py 5.3 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-09-01T105634.md 2.4 KB View
- results.json 2.5 KB View Raw