Fisher-Geometry Expert Menu

Integrate the method at the expert-selection and model-storage layer of a small mixture-of-experts classifier or a collection of LoRA adapters. First train M much larger than k candidate experts on different tasks, data shards, or bootstrap samples, then freeze them for the menu-construction experiment. On a calibration set, compute every candidate's predictive loss and score norm s(j,a), where s(j,a) is the squared norm of the gradient of log p_a(y_j given q_j) with respect to the candidate's parameters. Maintain an exponential moving average and set I_hat(q_j,a) = epsilon plus the moving average of s(j,a). Estimate the oracle candidate by selecting the expert with lowest validation loss for each context. Use b_hat(q_j)^2 equal to the squared gradient norm of the shared-trunk loss with respect to its representation, or use one if this sensitivity estimate is unavailable. Compute D_hat using the displayed formula. Select k experts by farthest-point initialization under D_hat, followed by alternating assignment of each calibration context to its lowest-distortion selected expert and replacement of each selected expert by the candidate that minimizes the maximum assigned distortion. Train a lightweight router to predict these assignments. The mathematical quantities are the distortion and the predicted k^(-2/r) frontier; Fisher estimates, oracle labels, and router training are empirical approximations. Start with CIFAR-100 or a small multi-task image benchmark, M = 64 LoRA adapters, and k in {4, 8, 16, 32}. Compare against random selection, parameter-space k-means, and magnitude-based adapter pruning. Measure validation loss, stored parameters, and inference FLOPs. Success means lower loss at equal k or an approximately linear log-frontier slope near -2/r.