Support-Graph Parallel Scheduler

Integrate the scheduler into inference graph construction, after tracing a modular model but before kernel compilation. Partition the model state into explicit wires: channel groups for grouped convolutions or MLPs, attention heads for head-local projections, and separate feature blocks for block-diagonal linear layers. Represent each primitive operation \(x\) by its wire support \(S_x\) and by its occurrence in each wire word \(w_i\). Initialize head pointers \(h_i=1\). At each scheduling step compute \(F(W;h)=\{x:\forall i\in S_x,(w_i)_{h_i}=x\}\); put every element of \(F\) into the next layer if their supports are pairwise disjoint or if a backend-certified fused kernel exists. Advance all pointers on the supports of selected operators and repeat until all words are consumed. If multiple feasible operations share wires, choose the subset maximizing estimated kernel occupancy, or greedily select the largest support-disjoint set. Preserve the original order for every pair whose supports overlap. The scheduler computes feasibility from the paper’s head rule; kernel launch cost, memory bandwidth, and fusion benefit must be measured empirically. First test a 12-layer grouped MLP and a 4-head Transformer with head-local feed-forward modules on an NVIDIA GPU, comparing sequential execution against this scheduler at batch sizes 1 and 32. Measure critical-path latency, number of kernel launches, peak activation memory, and exact output agreement. The expected signal is reduced inference latency and launch count, with no measurable change in logits or perplexity.