# Full-Rank Local Replay

- ID: 621
- Canonical URL: https://synthcore.org/idea/621/full-rank-local-replay
- API JSON: https://synthcore.org/api/idea/621.json
- API Markdown: https://synthcore.org/api/idea/621.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.09187](https://arxiv.org/abs/2607.09187)
- Category: training
- Solves: accuracy, stability, sample-efficiency
- ML areas: world-model, training-dynamics, regularization, optimizer
- Math tags: control-theory, optimization, linear-algebra, dynamical-systems
- Ratings: usefulness 7/10; difficulty 4/10; novelty 5/10

## Idea description

Train a neural state-space model using all replayed transitions, but assign larger weights to samples near the current operating context rather than discarding distant samples. Add a strictly positive weight floor so local adaptation cannot eliminate global coverage or make the regression problem rank-deficient. This should improve prediction across nonlinear regimes while retaining the numerical robustness of full-data training.

## Mathematical statement

Let z_j = (x_j,u_j) denote replay context j, let z_c be the current operating context, and let d_j be the distance d_j = sqrt((z_j-z_c)^T M (z_j-z_c)), where M is positive semidefinite. Define positive diagonal weights lambda_j = lambda_min + (lambda_max-lambda_min) exp(-d_j^2/(2 sigma^2)), with 0 < lambda_min <= lambda_max and sigma > 0. For a neural transition predictor f_theta, minimize the weighted empirical objective sum_j lambda_j ||f_theta(z_j)-x_{j+1}||^2 plus beta ||theta-theta_0||^2. In a linearized last-layer problem with feature matrix Phi and targets Y, the solution is W = (Phi^T Lambda Phi + beta I)^(-1) Phi^T Lambda Y, where Lambda = diag(lambda_1,...,lambda_N). Because Lambda is positive definite when lambda_min > 0 and beta > 0, the normal matrix is positive definite and has minimum eigenvalue at least beta.

## Key formulas

- lambda_j = lambda_min + (lambda_max-lambda_min) exp(-d_j^2/(2 sigma^2)),  d_j^2 = (z_j-z_c)^T M (z_j-z_c),  Lambda = diag(lambda_1,...,lambda_N).
- theta_plus = theta - eta grad_theta [ (1/N) sum_j lambda_j ||f_theta(z_j)-x_{j+1}||_2^2 + beta ||theta-theta_0||_2^2 ].
- W_star = (Phi^T Lambda Phi + beta I)^(-1) Phi^T Lambda Y,  Phi^T Lambda Phi + beta I >= beta I > 0.

## Implementation notes

1. Integration point: use this weighting rule in the replay loss of an online neural state-space model, latent world model, or locally adapted dynamics predictor. Store transitions (x_t,u_t,x_{t+1}) in a replay buffer, normalize state and input coordinates, and define z_t = [x_t;u_t]. At each update, define z_c from the latest observed context or from the mean context of the current rollout. 2. Pseudocode: sample a minibatch or the full buffer; compute d_j^2 = (z_j-z_c)^T M (z_j-z_c); compute lambda_j = lambda_min + (lambda_max-lambda_min) exp(-d_j^2/(2 sigma^2)); normalize weights to have minibatch mean one; compute L = mean_j lambda_j ||f_theta(z_j)-x_{j+1}||^2 + beta ||theta-theta_0||^2; backpropagate and update with SGD or Adam. Do not discard low-weight samples. 3. Taken directly from the paper's mechanism: positive diagonal weighting, operating-point localization, and retention of every data column. Estimated empirically: the metric M, bandwidth sigma, floor lambda_min, and ridge beta. Start with M equal to the identity, lambda_max = 1, lambda_min = 0.05, and sigma equal to the median pairwise context distance. 4. First cheap experiment: compare uniform replay, hard nearest-neighbor replay, and weighted replay on an MLP predicting a two-regime nonlinear dynamical system or the nonlinear two-tank system. Use the same buffer and optimizer, and measure one-step and 20-step rollout error near and far from the current context. The quantitative prediction is a smooth weight decay: at d = sigma, the Gaussian excess weight is exp(-1/2) = 0.6065 of its value at d = 0. The last-layer normal matrix should have minimum eigenvalue at least beta, while hard selection may become singular. Sweeping sigma should produce a U-shaped validation curve: small sigma causes global-regime forgetting, while excessively large sigma removes localization. With lambda_min below approximately 0.01, predict increased far-regime error; with sigma above roughly twice the median context distance, predict behavior close to uniform replay.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and ran a fixed-seed full-rank local replay MVP with Gaussian weighting, a positive weight floor, ridge-stabilized linear algebra checks, and a nonlinear MLP replay comparison. The quantitative predictions were confirmed: excess weight at d=sigma was 0.60653066 for every tested sigma, bandwidth decay matched exp(-d^2/(2 sigma^2)), and the ridged normal matrix had minimum eigenvalue 0.0300000000000 for beta=0.03. Localization improved near-context MSE versus uniform, 4.23e-05 versus 1.73e-04, but harmed far-context MSE, 6.63e-03 versus 5.75e-05, so this is a mechanism success and local-adaptation tradeoff rather than an overall accuracy win.

### Mechanism check

- Verdict: Built and ran a fixed-seed full-rank local replay MVP with Gaussian weighting, a positive weight floor, ridge-stabilized linear algebra checks, and a nonlinear MLP replay comparison. The quantitative predictions were confirmed: excess weight at d=sigma was 0.60653066 for every tested sigma, bandwidth decay matched exp(-d^2/(2 sigma^2)), and the ridged normal matrix had minimum eigenvalue 0.0300000000000 for beta=0.03. Localization improved near-context MSE versus uniform, 4.23e-05 versus 1.73e-04, but harmed far-context MSE, 6.63e-03 versus 5.75e-05, so this is a mechanism success and local-adaptation tradeoff rather than an overall accuracy win.
- Confidence: 9/10
- Limitations: Only one synthetic one-step regression system and one operating context were tested. No online changing-context rollout, multi-step benchmark, speed study, or systematic validation sweep of sigma and lambda_min was performed. A positive weight floor alone does not make Phi^T Lambda Phi full rank if Phi is rank-deficient; the positive-definiteness guarantee here depends on beta>0.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0224
- Paired wins: 1/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_local_replay.py](https://synthcore.org/code/278/bench_local_replay.py)
- [bench_report.json](https://synthcore.org/code/278/bench_report.json)
- [bench_report_rerun.json](https://synthcore.org/code/278/bench_report_rerun.json)
- [report.md](https://synthcore.org/code/278/report.md)
- [report_bench_2026-08-31T000213.md](https://synthcore.org/code/278/report_bench_2026-08-31T000213.md)
- [results.json](https://synthcore.org/code/278/results.json)
- [run_experiment.py](https://synthcore.org/code/278/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/278)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.