GP-DPP Adaptive Token Selection

Integrate this before an expensive attention or MIL block, operating on candidate token or patch embeddings X with shape N by d. Run a frozen teacher once to produce a scalar relevance r_i for every candidate and normalize r_i to [0,1]. Use an RBF kernel k(x,x')=exp(-||x-x'||^2/(2ell^2)) or a cosine kernel, with length scale ell estimated from median pairwise distances on a random subsample. Compute the GP posterior covariance k_D from a small inducing subset of teacher-labelled candidates, using Cholesky factorization rather than an explicit inverse. Greedy pseudocode: initialize S as empty; while |S|<B_max, compute each candidate's conditional variance v_i(S) from the Cholesky factor of K_D[S,S]+sigma_n^2 I, compute gain_i=alpha*r_i+0.5*beta*log(1+v_i(S)/sigma_n^2), choose i*=argmax gain_i, stop if gain_i*<=tau, otherwise append i* to S and update the Cholesky factor by a rank-one extension. Pass only X[S] to the student attention or MIL model and preserve original positions as positional metadata. Compute GP posterior and variance using the stated equations; estimate teacher scores, ell, sigma_n, alpha, beta, and tau empirically. First test ViT-Tiny on ImageNet-100 or a small histopathology MIL benchmark against top-k, uniform random, and k-center selection at equal retained-token counts. Measure accuracy, teacher-score retention, embedding coverage, wall-clock attention FLOPs, and peak activation memory. Success means a better accuracy-versus-token curve than top-k, at least 1.5x lower attention compute, and adaptive retention that decreases on easy examples without degrading hard-example accuracy.