Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training ML: Graph nn Math: Spectral theory Clear all

✗ Failed on benchmark 2026

Diffusion-DPP Gradient Batches

Replace uniform minibatch sampling by a fixed-size determinantal point process whose similarity matrix is a diffusion kernel on the training-data k-NN graph. The sampler repels nearby or redundant examples while preserving multiple diffusion modes, so a small batch should cover intrinsic data geometry and provide lower-variance estimates of losses and gradients.

Useful8/10

Difficulty6/10

Novelty5/10

Training Accuracy Sample efficiency Generalization

Paper: Fast determinantal sampling on general spaces and diffusion geometry arXiv:2607.06644

✓✓ Beats tuned baseline 2026

Laplacian-Coherence Graph Minibatches

Replace uniform node minibatches in a GNN with a coreset selected from a small random candidate set using local Laplacian-column coherence. Select nodes whose connectivity signatures are least redundant with already selected nodes, while retaining inverse-probability weights for unbiased loss estimates. This should improve coverage of weakly connected graph clusters and preserve smooth graph signals at the same batch size.

Useful7/10

Difficulty5/10

Novelty7/10

Training Sample efficiency Scalability Accuracy

Paper: Scalable Graph Coreset Selection via Greedy Sampling arXiv:2607.27602