Semantic-Aware Task Clustering for Constructive and Cooperative Multi-Tasking

What the math gives to ML

The paper provides a practical decomposition of joint multi-task representation learning into an inexpensive probe-training phase followed by a discrete task-grouping decision and within-group cooperative training. The transferable asset is not HDBSCAN or UMAP individually, which are standard tools, but the use of learned semantic representations to decide where parameter sharing is safe, thereby preventing negative transfer before expensive end-to-end training. This can be transplanted into multi-task Transformers or MoE models by learning short task embeddings, clustering them once, and replacing one globally shared trunk with cluster-specific trunks while retaining lightweight task heads. The paper's explicit objective also gives a useful criterion: clustering should improve the sum of task-wise information or its supervised variational proxy, rather than merely grouping tasks by dataset or label names.