2p+1 Random Fourier Dynamics Loss
Implementation & benchmark of arXiv:2607.16035 — Dynamic models with $p$ parameters are identified by $2p+1$ random features
Source paper: Dynamic models with $p$ parameters are identified by $2p+1$ random features arXiv:2607.16035 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Train a parametric neural dynamical model by matching randomized Fourier features of observed and simulated trajectory windows, using k=2p+1 features when the model has p trainable dynamic parameters. The random projections compress long noisy trajectories into a small identification signal while retaining nonlinear dependence on all lags, potentially making model calibration less sensitive to correlated, non-Gaussian, or state-dependent observation noise.
Formulas
Mathematical statement
The paper defines a length-(m+1) window x=(x_1,...,x_{m+1})^ op with x_j\in\mathbb{R}^d and draws k=2p+1 random features. For feature i and lag j, \Omega_{i,j}\overset{\mathrm{iid}}{\sim}N(0,I_d), while \alpha_i\overset{\mathrm{iid}}{\sim}U(-\pi,\pi), and the feature is \varphi_i(x)=\cos(\sum_{j=1}^{m+1}\Omega_{i,j}\cdot x_j+\alpha_i). Here p is the dimension of the unknown dynamic parameter vector heta, d is the observation dimension, m is the number of lags, and k is the number of random features. The adaptation matches empirical feature means between observed and simulated windows: \widehat{\mu}_i(W)=|W|^{-1}\sum_{x\in W}\varphi_i(x), and minimizes \mathcal{L}_{RF}( heta)=k^{-1}\sum_{i=1}^k(\widehat{\mu}_i(W_ heta)-\widehat{\mu}_i(W_{obs}))^2. The exact feature count 2p+1 is the paper's structural prescription; the squared empirical discrepancy is the neural-network training objective.
Implementation notes
(1) Integration point: use this as an auxiliary or replacement loss for a neural ODE, recurrent state-space model, or world model whose dynamics are controlled by a low-dimensional parameter vector theta in R^p. Keep the neural observation decoder unchanged. During training, simulate a predicted trajectory from the same initial state and construct overlapping windows of length m+1 from observed and predicted sequences. Apply the feature loss to normalized observable outputs rather than hidden states.
(2) Pseudocode:
Inputs: observed sequences Y, model f_theta, parameter dimension p, lag m
Set k = 2*p + 1
Sample and freeze Omega[i,j] ~ Normal(0, I_d), alpha[i] ~ Uniform(-pi, pi)
For each minibatch:
Yhat = rollout(f_theta, initial_states(Y), horizon=T)
Wobs = all_windows(Y, m+1)
What = all_windows(Yhat, m+1)
For i = 1,...,k:
aobs = mean over x in Wobs of cos(sum_j dot(Omega[i,j], x[j]) + alpha[i])
ahat = mean over x in What of cos(sum_j dot(Omega[i,j], x[j]) + alpha[i])
L_RF = mean_i (ahat - aobs)^2
L = L_RF + lambda * one_step_MSE
update theta with Adam or SGD
(3) The paper supplies the feature construction and feature budget k=2p+1. Estimate feature means empirically over minibatch windows. Normalize each observation coordinate using training-set mean and standard deviation; optionally multiply all frequencies by a bandwidth s, replacing Omega by s Omega, and tune s on validation data because the extracted text fixes unit Gaussian frequencies but does not specify scaling. Freeze the random features during training so the objective is stationary, and repeat over multiple random seeds.
(4) First experiment: train a 2- or 3-parameter neural ODE on noisy Lorenz-63 data or a synthetic damped oscillator with m=1 or m=2. Compare one-step MSE, Gaussian likelihood, and the proposed RF loss with k=2p+1, k=4p+2, and k=16 using identical rollouts and optimizers. Measure parameter recovery, long-horizon rollout error, and robustness to correlated Laplace observation noise. Success means equal-or-better parameter recovery and long-horizon prediction at comparable rollout cost, especially when noise is non-Gaussian or temporally correlated. Falsification is failure to beat MSE or likelihood at equal computation, or large degradation across random-feature seeds.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.