Unstable-Manifold-Aware Ensemble Averaging

1. Integration point: use this as an inference-time aggregation layer for an ensemble of RNNs, neural state-space models, or reservoir computers that all receive the same history and produce h-step state forecasts. Keep model parameters independently initialized, or generate diversity through bootstrap data subsets and dropout masks. For a cheap first version, do not backpropagate through the unstable-direction estimate. 2. Pseudocode: compute every forecast y_i=F_i^(h)(x_t); set m=(1/N) sum_i y_i; form the centered forecast matrix D=[y_1-m,...,y_N-m]; take the leading left singular vector of D as u, which estimates the dominant forecast-spread direction without computing Jacobians; calculate z_i=u^T(y_i-m); estimate validation kurtosis K_h of these coordinates; if K_h is below a chosen threshold K_0, return m, otherwise clip z_i at tau=3 validation standard deviations and replace the mean's unstable coordinate by the clipped mean. 3. Computed versus estimated: forecasts and singular vectors are computed online; independence, zero bias, and the N^{-1/2} law are assumptions to test; K_h, sigma_u, and tau are estimated on held-out trajectories. 4. First experiment: train 16 small GRUs or echo-state networks on Lorenz-63 or Mackey-Glass, evaluate horizons h=1,5,10,20, and compare the best single model, ordinary mean, validation-MSE weighting, and this estimator. Plot unstable-coordinate RMSE against ensemble size N=2,4,8,16. The predicted signature is a log-log slope near -1/2 in the light-tailed regime, with flattening or outliers at horizons where kurtosis grows; robust aggregation should reduce this flattening without harming short-horizon mean error.