Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

2 ideas found

Category: Training Solves: Memory ML: Graph nn Clear all

✗ Mechanism failed 2026

Two-level local/coarse GNN training

Partition a large graph into induced subgraphs and perform most parameter updates using only local subgraphs, interleaving them with inexpensive global updates on a randomly subsampled coarse graph. The coarse correction preserves information about cross-partition dependencies while reducing full-graph message passing and communication cost.

Useful8/10

Difficulty5/10

Novelty6/10

Training Speedup Scalability Memory

Paper: Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks arXiv:2608.22575

Unverified 2026

Hebbian SDR Adapter for Streaming Context Learning

Attach a local BCM-trained binary adapter to a pretrained or frozen encoder, allowing new graph nodes or streaming examples to acquire representations without backpropagating through the main network. The adapter learns only from positive co-occurrence statistics and maintains sparse codes, providing a low-memory continual-learning path.

Useful6/10

Difficulty4/10

Novelty7/10

Training Sample efficiency Stability Memory

Paper: Columnar-Embedder: A Biologically Inspired Cortical Architecture for Binary Sparse Distributed Graph Representations arXiv:2608.20408