Tree-Cone Distribution Head
Implementation & benchmark of arXiv:2607.02231 — Characterizing finite posets whose probabilistic powerdomain are RB-domains
Source paper: Characterizing finite posets whose probabilistic powerdomain are RB-domains arXiv:2607.02231 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Represent a neural network's categorical output over a rooted tree using cumulative probability mass on each rooted subtree. Train pairs of examples with a stochastic-dominance loss that compares these subtree masses, avoiding enumeration of all upper sets and making hierarchical monotonicity explicit. This is suitable for taxonomies, severity levels, hierarchical intents, and structured world-model states.
Formulas
Mathematical statement
For a finite poset P, a probability distribution is p=(p_x)_{x in P} in the nonnegative orthant with sum_x p_x=1, and stochastic order is p <= q if and only if p(U) <= q(U) for every upper set U contained in P, where p(U)=sum_{x in U} p_x. The paper proves that the probabilistic powerdomain has the relevant finite approximation structure exactly when P has a least element and its undirected Hasse graph is a tree. Root such a tree at its least element r, orienting edges away from r. For every non-root node v, let U_v={u in P: v <= u} be the rooted subtree above v and define the cumulative mass s_v(p)=p(U_v)=sum_{u in U_v} p_u. Every upper set is a disjoint union of rooted subtrees U_v corresponding to its minimal elements, so p <= q is equivalent to s_v(p) <= s_v(q) for every non-root v. The neural adaptation uses the differentiable penalty L_dom(p,q)=sum_{v != r} w_v [s_v(p)-s_v(q)]_+^2, where [z]_+=max(z,0), w_v are nonnegative weights, and p,q are network-predicted distributions. The tree relation s_v=p_v+sum_{c in child(v)} s_c exposes the local simplicial cone constraints.
Implementation notes
Integrate this as a distribution head and auxiliary loss for a model whose output classes are arranged as a rooted tree. First store the parent index, children list, depth, and a postorder traversal of the tree. The network produces logits z in R to the power of |P|; use p=softmax(z) for a minimal implementation, or replace this with a hierarchical conditional-softmax head if reducing output computation is important. For each batch item, compute subtree masses by reverse traversal: initialize s_v=p_v and then add s_v to its parent, so after the traversal s_v=sum_{u in U_v} p_u. Given a training pair (x_a,x_b) with a known desired order a <= b, compute p=f_theta(x_a) and q=f_theta(x_b), then add lambda_dom times sum_v w_v relu(s_v(p)-s_v(q)) squared to the ordinary cross-entropy. If the direction of the pair is uncertain, use a binary target t and apply relu(s_v(p)-s_v(q)) for t=1 and relu(s_v(q)-s_v(p)) for t=0. The theorem supplies the exact reduction from all upper sets to rooted-subtree coordinates; the weights, lambda_dom, and whether to use logits or a hierarchical head are empirical choices. Start with a small MLP or Transformer on synthetic tree-ordered classification data and then test on a real taxonomy dataset. Compare flat softmax plus pairwise class penalties, standard hierarchical softmax, and this head at equal parameter count. Success means fewer stochastic-order violations on held-out pairs, improved hierarchical accuracy or calibration at equal cross-entropy, and O(|P|) loss computation rather than exponential enumeration of upper sets.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.