{
 "artifacts": [
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1220/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1220/results.json"
  },
  {
   "name": "riccati_skip_experiment.py",
   "url": "https://synthcore.org/code/1220/riccati_skip_experiment.py"
  }
 ],
 "category": "dynamics",
 "description": "Add an uncertainty-aware observation scheduler to a neural state-space model or recurrent world model. Between expensive observation-encoder updates, propagate the latent state using the learned dynamics; periodically compute a decimated Riccati prediction and choose the largest skip length whose predicted covariance remains below a task-specific bound. This replaces a fixed observation stride with a principled, state-dynamics-dependent schedule.",
 "download_zip": "https://synthcore.org/download/1220",
 "formulas_latex": [
  "$$A_d=A^d,\\qquad Q_d=\\sum_{j=0}^{d-1}A^jQ(A^j)^\\mathsf{T}.$$",
  "$$P_d=A_dP_dA_d^\\mathsf{T}-A_dP_dH^\\mathsf{T}(HP_dH^\\mathsf{T}+R)^{-1}HP_dA_d^\\mathsf{T}+Q_d.$$",
  "$$d^*=\\max\\{d\\in\\{1,\\ldots,d_{\\max}\\}:g(P_d)\\leq\\tau\\},\\qquad g(P)=\\lambda_{\\max}(P)\\ \\text{or}\\ \\operatorname{tr}(P).$$",
  "$$P^-_{t+d}=A^dP^+_t(A^d)^\\mathsf{T}+\\sum_{j=0}^{d-1}A^jQ(A^j)^\\mathsf{T}.$$"
 ],
 "id": 3033,
 "implementation": "1. Exact integration point: use a recurrent state-space model with latent update z_(t+1) = f_theta(z_t,u_t) and an expensive observation pathway e_t = Encoder_phi(y_t). Run the observation pathway only when the scheduler requests an update; otherwise roll out f_theta for d steps. Maintain a diagonal covariance P for the first implementation, then test a low-rank-plus-diagonal version. 2. Pseudocode: at every scheduling time, evaluate the latent Jacobian A using automatic differentiation, estimate H from the observation head, and estimate Q from the moving covariance of one-step latent residuals z_(t+1) - f_theta(z_t,u_t). Estimate R from sensor residuals or learn a positive diagonal parameter. For each candidate d from 1 through d_max, compute A_d by repeated multiplication, accumulate Q_d using Q_d \u003c- Q_d + A^(d-1) Q (A^(d-1))^T, solve the Riccati equation by fixed-point iteration, and select the largest feasible d. During skipped steps, propagate P \u003c- A P A^T + Q. After an observation, apply P_plus \u003c- P - P H^T (H P H^T + R)^(-1) H P. 3. Taken directly from the paper are the equivalent matrices, covariance propagation rule, Riccati equation, and covariance-threshold decision. Estimated empirically are local Jacobians, Q, R, and the threshold tau; tau can be calibrated from validation prediction-error quantiles. 4. First cheap experiment: train a GRU or diagonal neural state-space model on Moving-MNIST or Lorenz-96 sequences, using an observation encoder at every frame as baseline. Compare fixed strides d = 1, 2, 4, and 8 against Riccati-gated skipping at equal latent dimensions. Measure encoder calls, rollout error, and empirical latent error covariance. The quantitative prediction is that the selected stride decreases sharply as the largest eigenvalue of A approaches one, and predicted versus empirical g(P_d) should agree within approximately 20 percent when local linearization is accurate. If measured g(P_d) exceeds tau, the next observation interval must use a smaller d.",
 "math_summary": "Let the latent state satisfy z_(t+1) = f_theta(z_t,u_t) + w_t, with observation y_t = h_theta(z_t) + v_t, process-noise covariance Q positive semidefinite, and observation-noise covariance R positive definite. Around the current latent trajectory, define A as the Jacobian of f_theta with respect to z and H as the Jacobian of h_theta with respect to z. For an integer skip factor d, the equivalent dynamics over d prediction steps are A_d = A^d and Q_d = sum from j=0 to d-1 of A^j Q (A^j)^T. The predicted steady-state prior covariance P_d is the stabilizing solution of the decimated discrete algebraic Riccati equation P_d = A_d P_d A_d^T - A_d P_d H^T (H P_d H^T + R)^(-1) H P_d A_d^T + Q_d. Choose the maximum d such that a scalar uncertainty functional g(P_d), such as the largest eigenvalue or trace, is no greater than a threshold tau. The construction assumes that the local pair (A_d,H) is detectable and that the neural dynamics are approximately linear over the skip interval.",
 "math_tags": [
  "control-theory",
  "dynamical-systems",
  "linear-algebra",
  "optimization"
 ],
 "ml_areas": [
  "ssm",
  "rnn",
  "world-model",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2609.02010",
  "arxiv_url": "https://arxiv.org/abs/2609.02010",
  "summary_what_math_gives_to_ml": "The paper provides a constructive covariance-growth mechanism for skipping intermediate observations: over a decimation interval of length d, the dynamics become A^d and accumulated process noise becomes Q_d = sum from j=0 to d-1 of A^j Q (A^j)^T. A single Riccati solve then predicts the steady-state uncertainty of the decimated estimator, enabling selection of the largest d satisfying a prescribed covariance bound. This transfers naturally to neural state-space models and recurrent world models by using a local Jacobian for A, learned residual covariance for Q, and an observation-head Jacobian for H. The resulting controller can adaptively skip encoder, sensor, or cross-attention updates while retaining a falsifiable uncertainty boundary.",
  "title": "Efficient Sensor Fusion Through Covariance-Constrained Observation Decimation (CCOD)",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "speedup",
  "stability",
  "scalability"
 ],
 "title": "Riccati-Gated Observation Skipping",
 "url": "https://synthcore.org/idea/3033/riccati-gated-observation-skipping",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a NumPy MVP implementing decimated dynamics/noise covariance, fixed-point Riccati prediction, covariance-threshold stride selection, and a nonlinear toy rollout. The core math was numerically validated: predicted versus empirical largest covariance eigenvalues differed by at most 0.41%, and uncertainty increased with skip length; near-unit dynamics reduced the selected maximum stride from 16 to 14. However, in the toy experiment the adaptive scheduler always selected stride 8, matching fixed stride 8 exactly, so no adaptive performance win or dynamic scheduling benefit was demonstrated.",
   "metrics": {
    "baseline": "Stride 1: RMSE 0.1647 +/- 0.0068, 240 encoder calls; fixed stride 4: RMSE 0.1939 +/- 0.0121, 60 calls; fixed stride 8: RMSE 0.2024 +/- 0.0134, 30 calls.",
    "idea": "Riccati adaptive: RMSE 0.2024 +/- 0.0134, 30 encoder calls, mean stride 8.0; identical to fixed stride 8 in this stationary toy setup. Math check maximum relative covariance error: 0.0041."
   },
   "how_to_run": "python3 riccati_skip_experiment.py",
   "files": [
    "riccati_skip_experiment.py",
    "results.json"
   ],
   "limitations": "This is a known-dynamics, two-dimensional toy system rather than a trained neural state-space model; Jacobian, Q, and R estimation were not learned. The dynamics and uncertainty were stationary, so the adaptive scheduler had no opportunity to vary its stride during a rollout. No GPU, encoder wall-clock timing, Moving-MNIST, or low-rank covariance variant was tested."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T12:35:18",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}