{
 "artifacts": [
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1068/bench_report.json"
  },
  {
   "name": "observability_bench.py",
   "url": "https://synthcore.org/code/1068/observability_bench.py"
  },
  {
   "name": "observability_experiment.py",
   "url": "https://synthcore.org/code/1068/observability_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1068/report.md"
  },
  {
   "name": "report_bench_2026-09-01T235430.md",
   "url": "https://synthcore.org/code/1068/report_bench_2026-09-01T235430.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1068/results.json"
  }
 ],
 "category": "dynamics",
 "description": "Add an observability objective to an RNN so that a finite trajectory of selected hidden coordinates preserves information about the initial hidden state. The regularizer maximizes the smallest singular value or log determinant of the finite-horizon observation Jacobian, counteracting ReLU activation masks that erase hidden-state directions.",
 "download_zip": "https://synthcore.org/download/1068",
 "formulas_latex": [
  "xₜ₊₁ = [W xₜ + U uₜ + b]₊,    yₜ = C xₜ",
  "Dₜ = diag(1{W xₜ + U uₜ + b \u003e 0}),    P₀ = I,    Pₜ₊₁ = Dₜ W Pₜ",
  "J_T = [C P₀; C P₁; …; C P_T],    G_T = J_Tᵀ J_T + εI",
  "L_obs = −log det(G_T),    m ≥ n/2 for the predicted global-observability threshold"
 ],
 "id": 2843,
 "implementation": "Integration point: use a vanilla ReLU RNN or a recurrent state-space model with transition xₜ₊₁ = [W xₜ + U uₜ + b]₊. Choose C as a fixed coordinate-selection matrix, or learn a soft selection matrix and discretize it after training. During each training batch, unroll a prefix of length T while retaining x₀. Pseudocode: set P = I; store J_blocks = [C]; for each step compute z = W x + U u + b, D = diag(z \u003e 0), update x = relu(z), update P = D W P, and append C P to J_blocks. Concatenate the blocks into J, form G = JᵀJ + εI, and optimize task_loss + λ times negative_logdet(G). For large n, estimate the smallest eigenvalue with Lanczos rather than constructing the full Jacobian. The paper supplies the mask-product Jacobian and the m ≥ n/2 lower-bound mechanism; the implementation estimates finite-batch Jacobians and conditioning empirically. First experiment: use an n = 16 ReLU RNN on permuted sequential MNIST or the copy-memory task, compare m = 2, 4, 7, 8, and 12, with and without the regularizer. Train an auxiliary decoder from y₀:T to reconstruct x₀. The quantitative prediction is a transition near m = 8 = n/2: below eight observed coordinates, increasing T should leave substantial hidden-state collisions, whereas at or above eight the smallest singular value and reconstruction accuracy should increase sharply. Measure whether the empirical transition lies within one observation node of n/2.",
 "math_summary": "Let the recurrent state be xₜ ∈ Rⁿ with dynamics xₜ₊₁ = [W xₜ + U uₜ + b]₊, where W and U are parameter matrices, uₜ is the input, b is a bias, and [·]₊ is componentwise ReLU. Let yₜ = C xₜ, where C ∈ Rᵐˣⁿ selects m observed coordinates. Define the finite-horizon observation map O_T(x₀) = [y₀; y₁; …; y_T]. Local observability requires the Jacobian J_T = ∂O_T/∂x₀ to have rank n. With Dₜ = diag(1{W xₜ + U uₜ + b \u003e 0}), the state-transition Jacobian is P₀ = I and Pₜ₊₁ = Dₜ W Pₜ, so J_T is the vertical stack of C P₀ through C P_T. The Gramian G_T = J_Tᵀ J_T + εI is well-conditioned when its smallest eigenvalue is large. The paper predicts that unrestricted real-valued ReLU networks require at least m ≥ n/2 observation nodes for global finite-horizon observability.",
 "math_tags": [
  "control-theory",
  "dynamical-systems",
  "linear-algebra",
  "spectral-theory"
 ],
 "ml_areas": [
  "rnn",
  "ssm",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.29650",
  "arxiv_url": "https://arxiv.org/abs/2608.29650",
  "summary_what_math_gives_to_ml": "The paper gives a transferable finite-horizon observability mechanism for recurrent networks. Its strongest result is that general real-valued ReLU dynamics can lose state information through activation masks, yielding a global lower bound of m ≥ n/2 observed hidden coordinates, while nonnegative ReLU trajectories reduce exactly to linear-system observability. A practical neural-network transfer is to regularize the finite-horizon observation Jacobian so that hidden-state information is preserved, and to use positivity constraints when a classical linear observability certificate is desirable.",
  "title": "On the Number of Observation Nodes in Recurrent Neural Networks with Linear Threshold and ReLU Functions",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 8
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Finite-Horizon Hidden-State Observability Regularizer",
 "url": "https://synthcore.org/idea/2843/finite-horizon-hidden-state-observability-regularizer",
 "verification": {
  "benchmark": {
   "track": "dynamics",
   "model": "rnn_small",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.019937668344937265,
    "idea_mean": 0.019944699306506664,
    "delta_mean": 0.000007030961569398642,
    "per_seed_diffs": [
     0.0000020265579223632812,
     0.000035801902413368225,
     0.000004191882908344269,
     -0.000009101815521717072,
     -0.000007625669240951538,
     -0.000024205539375543594,
     0.00005044788122177124,
     0.000004712492227554321
    ],
    "idea_wins": 3,
    "n_pairs": 8,
    "p_value": 0.45785,
    "mde": 0.000020486566875431273,
    "mde_rel_pct": 0.10275307283177568,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.006
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.0015
      },
      "mean": 0.33852236717939377
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 0.024998844717629254
     },
     {
      "cfg": {
       "lr": 0.006
      },
      "mean": 0.0168350946623832
     }
    ],
    "full": {
     "mean": 0.019937668344937265,
     "std": 0.013005039202094617,
     "per_seed": [
      0.03213806077837944,
      0.009990346617996693,
      0.010487839579582214,
      0.014724131673574448,
      0.04623213782906532,
      0.006986701861023903,
      0.011433426290750504,
      0.027508702129125595
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0.019944699306506664,
    "std": 0.013896786279171906,
    "per_seed": [
     0.032140087336301804,
     0.010026148520410061,
     0.010492031462490559,
     0.01471502985805273,
     0.04622451215982437,
     0.006962496321648359,
     0.011483874171972275,
     0.02751341462135315
    ],
    "n": 0
   },
   "mechanism_signature": {
    "baseline_trained_model": {
     "confirmed": true,
     "horizon_T": 1,
     "observed": [
      {
       "m": 8,
       "observed_logdet": -383.529296875,
       "observed_rank_T1": 16,
       "observed_smin": 0.03744511306285858,
       "predicted_rank_upper_bound_T1": 16
      },
      {
       "m": 16,
       "observed_logdet": -327.2057189941406,
       "observed_rank_T1": 32,
       "observed_smin": 0.028290288522839546,
       "predicted_rank_upper_bound_T1": 32
      },
      {
       "m": 32,
       "observed_logdet": -227.1643829345703,
       "observed_rank_T1": 64,
       "observed_smin": 0.00004912145959679037,
       "predicted_rank_upper_bound_T1": 64
      }
     ],
     "predicted_counting_threshold_m": 32,
     "state_dim": 64
    },
    "idea_trained_model": {
     "confirmed": true,
     "horizon_T": 1,
     "observed": [
      {
       "m": 8,
       "observed_logdet": -381.6462097167969,
       "observed_rank_T1": 16,
       "observed_smin": 0.04268581420183182,
       "predicted_rank_upper_bound_T1": 16
      },
      {
       "m": 16,
       "observed_logdet": -325.7181701660156,
       "observed_rank_T1": 32,
       "observed_smin": 0.02824431285262108,
       "predicted_rank_upper_bound_T1": 32
      },
      {
       "m": 32,
       "observed_logdet": -225.09347534179688,
       "observed_rank_T1": 64,
       "observed_smin": 0.0018923624884337187,
       "predicted_rank_upper_bound_T1": 64
      }
     ],
     "predicted_counting_threshold_m": 32,
     "state_dim": 64
    },
    "prediction": "For T=1, m \u003e= n/2 is the counting threshold for full rank."
   },
   "custom_track": null
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 7,
   "verdict": "Built the finite-horizon ReLU observability Jacobian, Gramian metrics, exact autograd check, dimension sweep, horizon sweep, and a small log-det optimization control. The mask-product formula is numerically correct; full-rank onset occurred at m=9 (predicted counting boundary m=8) and T=4 for m=4 (predicted T=3), both within one of the theoretical thresholds. The regularizer raised logdet from -25.53 to -17.04 at m=9, but the smallest singular value remained near zero and numerical rank did not improve, so it is a promising conditioning signal rather than a demonstrated robust observability win.",
   "metrics": {
    "baseline": "m=9 fixed control: logdet -25.53, smin 6.86e-10, rank 15",
    "idea": "m=9 log-det optimization: logdet -17.04, smin 2.56e-8, rank 14; autograd Jacobian max error 9.54e-7; observed boundaries m=9 and T=4"
   },
   "how_to_run": "python3 observability_experiment.py",
   "files": [
    "observability_experiment.py",
    "results.json"
   ],
   "limitations": "No sequential MNIST/copy-memory task, auxiliary decoder, FLOP/speed study, multi-seed statistics, learned observation matrix, or comparison against a trained vanilla RNN was tested. The optimization comparison is a small synthetic parameter-only control, and log-det can improve through singular-value scaling without eliminating rank deficiency."
  },
  "status": "failed_benchmark",
  "status_label": "Failed on benchmark",
  "updated_at": "2026-09-01T23:54:30",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}