Excursion-Adaptive Temporal Tokenization
Implementation & benchmark of arXiv:2608.07879 — Sharp Wasserstein Convergence Rates for Empirical Path Laws of Itô Processes
Source paper: Sharp Wasserstein Convergence Rates for Empirical Path Laws of Itô Processes arXiv:2608.07879 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a uniformly sampled trajectory sequence by a binary temporal partition whose intervals are split only when the observed trajectory makes an excursion larger than a threshold. Encode one summary token per retained leaf, optionally including duration and endpoint displacement, so smooth trajectory regions receive fewer tokens while rapidly changing regions retain resolution.
Formulas
Mathematical statement
The paper constructs a forest by recursively splitting a dyadic interval I=[u,v] with left endpoint u whenever sup_{t in I} ||Y_t-Y_u||>delta, and retaining I as a leaf otherwise. Here Y is a continuous path, delta>0 is the excursion threshold, and ||.|| is the Euclidean norm. For an Ito decomposition Y_t=Y_u+(drift increment)+(M_t-M_u), with drift bounded by G, martingale part M, and dyadic interval length h_l at level l, the paper uses ||Y_t-Y_s|| <= G|t-s|+||M_t-M_s|| and chooses a base level n_0 so that G h_l <= delta/4, yielding {sup_{t in I}||Y_t-Y_u||>delta} subset {sup_{t in I}||M_t-M_u||>3delta/4}. The quadratic-variation bound <M^i>_t-<M^i>_u <= G^2(t-u) <= G^2 h_l explains why large excursions become rare on fine intervals. The paper's broader path quantization scale is (log N)^(-1/2) under the supremum metric; the neural adaptation uses the forest as a variable-rate path approximation rather than estimating Wasserstein distance.
Implementation notes
Integrate this at the temporal tokenizer before a trajectory Transformer, temporal U-Net, or diffusion model. Inputs are a sampled trajectory y[0:T], timestamps, an excursion threshold delta, and a maximum tree depth L. Start with queue=[(0,T,0)]. For each interval (a,b,l), compute e=max_{a<=k<=b} ||y[k]-y[a]||. If e>delta, l<L, and b-a>1, split at m=floor((a+b)/2) and push (a,m,l+1) and (m,b,l+1); otherwise retain the interval as a leaf. For every leaf create a token containing y[a], y[b], normalized duration (b-a)/T, displacement y[b]-y[a], excursion e, and a learned depth embedding. Add relative-time attention biases using interval centers and durations. The paper's split rule is used directly; the drift and martingale inequalities motivate why fine intervals should rarely split but are not required by the implementation. Choose delta by a validation sweep targeting a fixed token budget, or use delta equal to a percentile of training-set one-step excursions. Enforce both a minimum interval length and a hard leaf cap. Train with random threshold jitter to reduce boundary sensitivity. First test on Lorenz-63, human-motion trajectories, and short Navier-Stokes rollouts using a 6-layer Transformer, comparing against uniform subsampling at matched token count and FLOPs. Measure token count, wall-clock attention cost, peak memory, one-step error, and long-horizon rollout error. Success is at least 1.5x fewer tokens at equal error, or lower rollout error at equal attention FLOPs.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.