Slow Contextual Worst-Case Curriculum

1. Integration point: place the sampler outside the existing training loop. For PPO, DQN, supervised learning, or diffusion training, define $K$ finite contexts such as simulator demand profiles, layouts, transition-noise levels, image corruptions, or domain styles. Maintain a probability vector $p(k)$, or a contextual bandit head producing logits from context features. Keep the learner architecture and optimizer unchanged. 2. Pseudocode: initialize $p_k=1/K$; sample context $k\sim p$; collect a minibatch or rollout under context $k$; perform standard learner updates using loss $L_k(\theta)$; evaluate held-out loss for every context, or estimate the sampled context with importance weighting; update an exponential moving average $m_k\leftarrow(1-\gamma)m_k+\gamma\widehat J(k)$; then set $p_k\leftarrow p_k\exp(\beta m_k)$ and renormalize. Mix with uniform probability, $p\leftarrow(1-\epsilon)p+\epsilon/K$, to preserve exploration. 3. Computed versus estimated: simplex normalization and exponential weighting are exact; context losses, returns, and contextual features are empirical estimates. Update the adversary only every $M=50$ to $500$ learner steps. 4. First cheap experiment: train PPO on a procedurally generated grid-world with eight layouts or transition-noise contexts, comparing uniform sampling, static domain randomization, and this method under identical learner budgets. Evaluate mean and maximum episodic loss on all contexts. The quantitative prediction is a two-timescale boundary: as $\beta/\alpha$ rises from about $0.01$ toward $0.1$ or larger, adversarial probabilities and worst-context loss should begin oscillating rather than converge smoothly. In the stable regime, maximum-context loss should decrease and $p_k$ should concentrate on initially difficult contexts before flattening as losses equalize. Accept the method only if maximum-context loss falls by at least 20% with no more than 10% mean-return degradation, and report the empirically measured oscillation onset versus $\beta$.