An efficient algorithm for testing the compatibility of phylogenies with nested taxa

Background Semi-labeled trees generalize ordinary phylogenetic trees, allowing internal nodes to be labeled by higher-order taxa. Taxonomies are examples of semi-labeled trees. Suppose we are given collection \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}$$\end{document}P of semi-labeled trees over various subsets of a set of taxa. The ancestral compatibility problem asks whether there is a semi-labeled tree that respects the clusterings and the ancestor/descendant relationships implied by the trees in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}$$\end{document}P. The running time and space usage of the best previous algorithm for testing ancestral compatibility depend on the degrees of the nodes in the trees in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}$$\end{document}P. Results We give a algorithm for the ancestral compatibility problem that runs in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(M_{\mathcal {P}}\log ^2 M_{\mathcal {P}})$$\end{document}O(MPlog2MP) time and uses \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(M_{\mathcal {P}})$$\end{document}O(MP) space, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M_{\mathcal {P}}$$\end{document}MP is the total number of nodes and edges in the trees in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}$$\end{document}P. Conclusions Taxonomies enable researchers to expand greatly the taxonomic coverage of their phylogenetic analyses. The running time of our method does not depend on the degrees of the nodes in the trees in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {P}$$\end{document}P. This characteristic is important when taxonomies—which can have nodes of high degree—are used.


Introduction
In the tree compatibility problem, we are given a collection P = {T 1 , T 2 , . . . , T k } of rooted phylogenetic trees with partially overlapping taxon sets. P is called a profile and the trees in P are the input trees. The question is whether there exists a tree T whose taxon set is the union of the taxon sets of the input trees, such that T exhibits the clusterings implied by the input trees. That is, if two taxa are together in a subtree of some input tree, then they must also be together in some subtree of T . The tree compatibility problem has been studied for over three decades [1][2][3][4].
In the original version of the tree compatibility problem, only the leaves of the input trees are labeled. Here we study a generalization, called ancestral compatibility, in which taxa may be nested. That is, the internal nodes may also be labeled; these labels represent higher-order taxa, which are, in effect, sets of taxa. Thus, for example, an input tree may contain the taxon Glycine max (soybean) nested within a subtree whose root is labeled Fabaceae (the legumes), itself nested within an Angiosperm subtree. Note that leaves themselves may be labeled by higher-order taxa. The question now is whether there is a tree T whose taxon set is the union of the taxon sets of the input trees, such that T exhibits not only the clusterings among the taxa, but also the ancestor/descendant relationships among taxa in the input trees. Our main result is a O(M P log 2 M P ) algorithm for the compatibility problem for trees with nested taxa, where M P is the total number of nodes and edges in the trees in P.

Background
The tree compatibility problem is a basic special case of the supertree problem. A supertree method is a way to synthesize a collection of phylogenetic trees with partially overlapping taxon sets into a single supertree that represents the information in the input trees. The supertree approach, proposed in the early 90s [5,6], has been used successfully to build large-scale phylogenies [7].
The original supertree methods were limited to input trees where only the leaves are labeled. Page [8] was among the first to note the need to handle phylogenies where internal nodes are labeled, and taxa are nested. A major motivation is the desire to incorporate taxonomies as input trees in large-scale supertree analyses, as way to circumvent one of the obstacles to building comprehensive phylogenies: the limited taxonomic overlap among different phylogenetic studies [9]. Taxonomies group organisms according to a system of taxonomic rank (e.g., family, genus, and species); two examples are the NCBI taxonomy [10] and the Angiosperm taxonomy [11]. Taxonomies spanning a broad range of taxa provide structure and completeness that might be hard to obtain otherwise. A recent example of the utility of taxonomies is the Open Tree of Life, a draft phylogeny for over 2.3 million species [12].
Taxonomies are not, strictly speaking, phylogenies. In particular, their internal nodes and some of their leaves are labeled with higher-order taxa. Nevertheless, taxonomies have many of the same mathematical characteristics as phylogenies. Indeed, both phylogenies and taxonomies are semi-labeled trees [13,14]. We will use this term throughout the rest of the paper to refer to trees with nested taxa.
The fastest previous algorithm for testing ancestral compatibility, based on earlier work by Daniel and Semple [15], is due to Berry and Semple [16]. Their algorithm runs in O log 2 n · τ P time using O(τ P ) space. Here, n is the number of distinct taxa in P and is the set of internal nodes of T i , for each i ∈ {1, . . . , k}, and d(v) is the degree of node v. While the algorithm is polynomial, its dependence on node degrees is problematic: semilabeled trees can be highly unresolved (i.e., contain nodes of high degree), especially if they are taxonomies.

Our contributions
As stated earlier, our main result is an algorithm to test ancestral compatibility that runs in O(M P log 2 M P ) time, using O(M P ) space. These bounds are independent of the degrees of the nodes of the input trees, a valuable characteristic for large datasets that include taxonomies. To achieve our result, we extend ideas from our recent algorithm for testing the compatibility of ordinary phylogenetic trees [2]. As in that algorithm, a central notion in the current paper is the display graph of profile P, denoted H P . This is the graph obtained from the disjoint union of the trees in P by identifying nodes that have the same label (see the section titled "Testing ancestral compatibility"). The term "display graph" was introduced by Bryant and Lagergren [17], but similar ideas have been used elsewhere. In particular, the display graph is closely related to Berry and Semple's restricted descendancy graph [16], a mixed graph whose directed edges correspond to the (undirected) edges of H P and whose undirected edges have no correspondence in H P . The second kind of edges are the major component of the τ P term in the time and space complexity of Berry and Semple's algorithm. The absence of such edges makes H P significantly smaller than the restricted descendancy graph. Display graphs also bear some relation to tree alignment graphs [18].
Here, we exploit the display graph more extensively than in our previous work. Although the display graph of a collection of semi-labeled trees is more complex than that of a collection of ordinary phylogenies, we are able to extend several of the key ideas-notably, that of a semiuniversal label-to the general setting of semi-labeled trees. As in [2], the implementation relies on a dynamic graph data structure, but it requires a more careful amortized analysis based on a weighing scheme.

Contents
This paper has five sections, in addition to this introduction. The section titled "Preliminaries" presents basic definitions regarding graphs, semi-labeled trees, and ancestral compatibility. The section titled "The display graph" introduces the display graph and discusses its properties. The section titled "Testing ancestral compatibility" presents BuildNT, our algorithm for testing ancestral compatibility. We first present the algorithm recursively, and then show how to transform it into an iterative algorithm, BuildNT N , that is easier to implement. We also give an example of the execution of BuildNT N . The "Implementation" section gives the implementation details for BuildNT N . The "Discussion" section gives some concluding remarks.

Graph notation
Let G be a graph. V(G) and E(G) denote the node and edge sets of G. The degree of a node v ∈ V (G) is the number of edges incident on v. A tree is an acyclic connected graph. In this paper, all trees are assumed to be rooted. For a tree T, r(T) denotes the root of T.
and u ≤ T v, then u is the parent of v and v is a child of u. If neither u ≤ T v nor v ≤ T u hold, then we write u T v and say that u and v are not comparable in T.

Semi-labeled trees
A semi-labeled tree is a pair T = (T , φ) where T is a tree and φ is a mapping from a set L(T ) to V(T) such that, for every node v ∈ V (T ) of degree at most two, v ∈ φ(L(T )) . L(T ) is the label set of T and φ is the labeling function of T .
For every node v ∈ V (T ), φ −1 (v) denotes the (possibly empty) subset of L(T ) whose elements map into v; these elements as the labels of v (thus, each label is a taxon). If Note that, by definition, every leaf in a semi-labeled tree is labeled. Further, any node, including the root, that has a single child must be labeled. Nodes with two or more children may be labeled or unlabeled. A semi-labeled tree T = (T , φ) is singularly labeled if every node in T has at most one label; T is fully labeled if every node in T is labeled.
Semi-labeled trees, also known as X-trees, generalize ordinary phylogenetic trees, also known as phylogenetic X-trees [14]. An ordinary phylogenetic tree is a semilabeled tree T = (T , φ) where r(T) has degree at least two and φ is a bijection from L(T ) into leaf set of T (thus, internal nodes are not labeled).
Let T = (T , φ) be a semi-labeled tree and let ℓ and ℓ ′ be two labels in L(T ). If φ(ℓ) ≤ T φ(ℓ ′ ), then we write ℓ ≤ T ℓ ′ , and say that ℓ ′ is a descendant of ℓ in T and that ℓ is an ancestor of ℓ ′ . We write , then we write ℓ � T ℓ ′ and say that ℓ and ℓ ′ are not comparable in T . If T is fully labeled and φ(ℓ) is the parent of φ(ℓ ′ ) in T, then ℓ is the parent of ℓ ′ in T and ℓ ′ is a child of ℓ in T ; two labels with the same parent are siblings.
Let T = (T , φ) be a semi-labeled tree. For each u ∈ V (T ), X(u) denotes the set of all labels in the subtree of T rooted at u; that is, X(u) = v:u≤ T v φ −1 (v). X(u) is called a cluster of T. Cl(T ) denotes the set of all clusters of T . It is well known [14, Theorem 3.5.2] that a semilabeled tree T is completely determined by Cl(T ). That is, if Cl(T ) = Cl(T ′ ) for some other semi-labeled tree T ′ , then T is isomorphic to T ′ .
Suppose A ⊆ L(T ) for a semi-labeled tree T = (T , φ). The restriction of T to A, denoted T |A, is the semi-labeled tree whose cluster set is Cl(T |A) = {X ∩ A : X ∈ Cl(T ) and X ∩ A � = ∅}. Intuitively, T |A is obtained from the minimal rooted subtree of T that connects the nodes in φ(A) by suppressing all vertices of degree two that are not in φ(A).
For a semi-labelled tree T , let us define D(T ) and N (T ) as follows.
Note that D(T ) consists of ordered pairs, while N (T ) consists of unordered pairs.

Profiles and ancestral compatibility
Throughout the rest of this paper We refer to P as a profile, and write L(P) to denote i∈[k] L(T i ), the label set of P. Figure 1 shows a profile where P is ancestrally compatible if there is a rooted semilabeled tree T that ancestrally displays each of the trees in P. If T exists, we say that T ancestrally displays P (see Fig. 2).
Given a subset X of L(P), the restriction of P to X, denoted P|X, is the profile defined as The proof of the following lemma is straightforward.
Lemma 2 Suppose P is ancestrally compatible and let T be a tree that ancestrally displays P. Then, for any X ⊆ L(P), T |X ancestrally displays P|X. For technical reasons, fully labeled trees are easier to handle than those that are not. Suppose P contains trees that are not fully labeled. We can convert P into an equivalent profile P ′ of fully-labeled trees as follows.
where ℓ is a distinct element from L ′ . We refer to P ′ as the profile obtained by adding distinct new labels to P (see Fig. 1). [15]) Let P ′ be the profile obtained by adding distinct new labels to P. Then, P is ancestrally compatible if and only if P ′ is ancestrally compatible. Further, if T is a semi-labeled phylogenetic tree that ancestrally displays P ′ , then T ancestrally displays P.

Lemma 3 (Daniel and Semple
From this point forward, we make the following assumption.

Assumption 1 For each i ∈ [k]
, T i is fully and singularly labeled.
By Lemma 3, no generality is lost in assuming that all trees in P are fully labeled. The assumption that the trees are singularly labeled is inessential; it is only for clarity. Note that, even with the latter assumption, a tree that ancestrally displays P is not necessarily singularly labeled. Figure 2 illustrates this fact.

The display graph
The display graph of a profile P, denoted H P , is the graph obtained from the disjoint union of the underlying trees T 1 , T 2 , . . . , T k by identifying nodes that have the same label. Multiple edges between the same pair of nodes are replaced by a single edge. See Fig. 3.
H P has O(M P ) nodes and edges, and can be constructed in O(M P ) time. By Assumption 1, there is a bijection between the labels in L(P) and the nodes of H P . Thus, from this point forward, we refer to the nodes of H P by their labels. It is easy to see that if H P is not connected, then P decomposes into label-disjoint subprofiles, and that P is compatible if and only if each subprofile is compatible. Thus, without loss of generality, we shall assume the following.
Assumption 2 H P is connected.

Positions
Our compatibility algorithm processes the trees in P from the top down, starting at the roots. We refer to the set of nodes in P currently being considered as a "position". The algorithm advances from the current position to the next by replacing certain nodes in the current position by their children. Formally, a position (for P) is a vector . Since labels may be shared among trees, we may have

Lemma 4 For any valid position U,
Proof By (V2), we have that T i |Desc i (U) and The lemma then follows from the definition of P|Desc P (U) .
For any valid position U, H P (U) denotes the subgraph of H P induced by Desc P (U). A valid position of special interest to us is U root , the root position, defined as follows.
That is, for each i ∈ [k], U root (i) is a singleton containing only the label of r(T i ). In Fig. 3,

Semi-universal labels
Let U be a valid position, and let ℓ be a label in U. Fig. 3, labels 1 and 2 are semi-universal in U root , but g is not, since g is in both L(T 2 ) and L( The term "semi-universal", borrowed from Pe'er et al. [19], derives from the following fact. Suppose that P is ancestrally compatible, that T is a tree that ancestrally displays P, and that ℓ is a semi-universal label for some valid position U. Then, as we shall see, ℓ must label the root u ℓ of a subtree of T that contains all the descendants of ℓ in T i , for every i such that ℓ ∈ L(T i ). The qualifier "semi" is because this subtree may also contain labels that do not descend from ℓ in any input tree, but descend instead from some other semi-universal label ℓ ′ in U. In this case, ℓ ′ also labels u ℓ . We exploit this property of semi-universal labels in our ancestral compatibility algorithm and its proof of correctness (see "Testing ancestral compatibility").
For each label ℓ ∈ L(P), let k ℓ denote the number of input trees that contain label ℓ. We can obtain k ℓ for every ℓ ∈ L(P) in O(M P ) time during the construction of H P .

Successor positions
For every i ∈ [k] and every ℓ ∈ L(T i ), let Ch i (ℓ) denote the set of children of ℓ in L(T i ). (1) In Fig. 3, the set of semi-universal labels in U root is Proof It suffices to argue that U ′ satisfies conditions (V1) and (V2). The lemma then follows from the fact that the connected components of H P (U ′ ) are label-disjoint.
U ′ must satisfy condition (V1), since U does. Suppose ℓ ∈ S. Then, for each i ∈ [k] such that ℓ ∈ L(T i ), Thus, since (V2) holds for U, it also holds for U ′ .

Overview of the algorithm
BuildNT (Algorithm 1) is our algorithm for testing compatibility of semi-labeled trees. Its argument, U, is a valid position in P such that H P (U) is connected. BuildNT relies on the fact-proved later, in Theorem 1-that if P|Desc P (U) is compatible, then U must contain a nonempty set S of semi-universal labels. If such a set S exists, the algorithm replaces U by its successor U ′ with respect to S. It then processes each connected component of H P (U ′ ) recursively, to determine if the associated subprofile is compatible. If all the recursive calls are successful, then their results are combined into a supertree for P|Desc P (U).
Page 6 of 12 Deng and Fernández-Baca Algorithms Mol Biol (2017) 12:7 In detail, BuildNT proceeds as follows. Line 1 computes the set S of semi-universal labels in U. If S is empty, then, P|Desc P (U) is incompatible, and, thus, so is P. This fact is reported in Line 3. Line 4 creates a tentative root r U , labeled by S, for the tree T U for L(U). Line 5 checks if S contains exactly one label ℓ, with no proper descendants. If so, by the connectivity assumption, ℓ must be the sole member of Desc P (U); that is, L(U) = ℓ . Therefore, Line 6 simply returns the tree with a single node, labeled by S = {ℓ}. Line 7 updates U, replacing it by its successor with respect to S. Let W 1 , W 2 , . . . , W p be the connected components of H P (U) after updating U. By Lemma 6, U |W j is a valid position, for each j ∈ [p]. Lines 8-12 recursively invoke BuildNT on U |W j for each j ∈ [p], to determine if there is a tree t j that ancestrally displays P|Desc P (U ∩ W j ). If any subproblem is incompatible, Line 12 reports that P is incompatible. Otherwise, Line 13 returns the tree obtained by making the t j s the subtrees of root r U .
Next, we argue the correctness of BuildNT.

Lemma 7 Let U be a valid position in P. If BuildNT(U ) returns a tree T U , then T U is a phylogenetic tree such that L(T U ) = L(U).
Proof We use induction on |L(U)|. The base case, where |L(U )| = 1, is handled by Lines 5-6. In this case, S = L(U) = {ℓ} and BuildNT(U ) correctly returns the tree consisting of a single node, labeled by {ℓ}. Otherwise, let W 1 , . . . , W p be the connected components of H P (U) in step 8. Since BuildNT(U ) returns tree T U , it must be the case that, for each j ∈ [p], the result t j returned by the recursive call to BuildNT(U |W j ) in Line 10 is a tree. Since |S| ≥ 1, we have |L(W j )| < |L(U )|, for each j ∈ [p].
Thus, we can assume inductively that t j is a phylogenetic tree for L(W j ). Since S ∪ j∈[p] L(W j ) = L(U ), the tree returned in Line 13 is a phylogeny with species set L(U).
Theorem 1 Let P = {T 1 , T 2 , . . . , T k } be a profile and let U root be the root position, as defined in Eq. (1). Then, BuildNT(U root ) returns either (i) a semi-labeled tree T that ancestrally displays P, if P is ancestrally compatible, or (ii) incompatible otherwise.
Proof BuildNT(U root ) either returns a tree or incompatible. We consider each case separately. . Consider any (ℓ, ℓ ′ ) ∈ D(T i ). Then, ℓ has a child ℓ ′′ in T i such that ℓ ′′ ≤ T i ℓ ′ -note that we may have ℓ ′′ = ℓ. There must be a recursive call to BuildNT(U ), for some valid position U, where ℓ is the set S of semi-universal labels obtained in Line 1. By Observation 2, label ℓ ′′ , and thus ℓ ′ , both lie in one of the connected components of the graph obtained by deleting all labels in S, including ℓ, and their incident edges from H P (U). It now follows from the construction of T that (ℓ, ℓ ′ ) ∈ D(T ). Thus, D(T i ) ⊆ D(T ). Now, consider any {ℓ, ℓ ′ } ∈ N (T i ). Let v be the lowest common ancestor of φ i (ℓ) and φ i (ℓ ′ ) in T i and let ℓ v be the label of v. Then, ℓ v has a pair of children, ℓ 1 and ℓ 2 say, in T i such that ℓ 1 ≤ T i ℓ, and ℓ 2 ≤ T i ℓ ′ . Because BuildNT(U root ) returns a tree, there are recursive calls BuildNT(U 1 ) and BuildNT(U 2 ) for valid positions U 1 and U 2 such that ℓ 1 is semi-universal for U 1 and ℓ 2 is semi-universal for U 2 . We must have U 1 � = U 2 ; other-wise, |U 1 (i)| = |U 2 (i)| ≥ 2, and, thus, neither ℓ 1 nor ℓ 2 is semi-universal, a contradiction. Further, it follows from the construction of T that we must have Desc P (U 1 ) ∩ Desc P (U 2 ) = ∅. Hence, ℓ � T ℓ ′ , and, therefore, {ℓ, ℓ ′ } ∈ N (T ). (ii) Asssume, by way of contradiction, that BuildNT(U root ) returns incompatible, but that P is ancestrally compatible. By assumption, there exists a semi-labeled tree T that ancestrally displays P. Since BuildNT(U root ) returns incompatible, there is a recursive call to BuildNT(U ) for some valid position U such that U has no semi-universal label, and the set S of Line 1 is empty. By Lemma 2, T |Desc P (U) ancestrally displays P|Desc P (U). Thus, by Lemma 4, T |Desc P (U) ancestrally displays T i |Desc i (U) , for every i ∈ [k]. Let ℓ be any label in the label set of the root of T |Desc P (U). Then, for each i ∈ [k] such that ℓ ∈ L(T i ), ℓ must be the label of the root of T i |Desc i (U). Thus, for each such i, U (i) = {ℓ}. Hence, ℓ is semi-universal in U, a contradiction.
L(U) in the supertree. The body of the loop closely follows the steps performed by a call to BuildNT(U ). Line 5 computes the set S of semi-universal labels in U. If S is empty, the algorithm reports that P is incompatible and terminates (Lines 6-7). The algorithm then creates a tentative root r U labeled by S for the tree T U for L(U), and links r U to its parent (Line 8). If S consists of exactly one element that has no proper descendants, we skip the rest of the current iteration of the while loop, and continue to the next iteration (Lines 9-10). Line 11 replaces U by its successor with respect to S. Lines 13-14 enqueue each of U |W 1 , U |W 2 , . . . , U |W p , along with r U , for processing in a subsequent iteration. If the while loop terminates without any incompatibility being detected, the algorithm returns the tree with root r U root .
Although the order in which BuildNT N processes connected components differs from that of BuildNTbreadth-first instead of depth-first-, it is straightforward

An iterative version
We now present BuildNT N (Algorithm 2), an iterative version of BuildNT, which lends itself naturally to an efficient implementation. BuildNT N performs a breadthfirst traversal of BuildNT's recursion tree, using a first-in first-out queue Q that stores pairs of the form (U, pred), where U is a valid position in P and pred is a reference to the parent of the node corresponding to U in the supertree built so far. BuildNT N simulates recursive calls in BuildNT by enqueuing pairs corresponding to subproblems. We explain this in more detail next.
BuildNT N initializes its queue to contain the starting position, U root , with a null parent. It then proceeds to the while loop of Lines 3-14. Each iteration of the loop starts by dequeuing a valid position U, along with a reference pred to the potential parent for the subtree for to see that the effect is equivalent, and the proof of correctness of BuildNT (Theorem 1) applies to BuildNT N as well. We thus state the following without proof. Theorem 2 Let P = {T 1 , T 2 , . . . , T k } be a profile. Then, BuildNT N (P) returns either (i) a semi-labeled tree T that ancestrally displays P, if P is ancestrally compatible, or (ii) incompatible otherwise.
Let Q be BuildNT N 's first-in first-out queue. In the rest of the paper, we will say that a valid position U is in Q if (U, pred) ∈ Q, for some pred. Let H Q be the subgraph of H P induced by {Desc(U) : U is in Q}. By Observation 1, H Q is obtained from H P through edge and node deletions.

Lemma 8 At the start of any iteration of BuildNT N 's while loop, the set of connected components of H Q is {V (H P (U)) : U is in Q}.
Proof The property holds at the outset, since, by Assumption 2, H P = H P (U root ) is a connected graph, and the only element of Q is (U root , null). Assume that the property holds at the beginning of iteration l. Let (U, pred) be the element dequeued from Q in Line 4. Then, H P (U) is connected. In place of (U, pred), Lines 13-14 enqueue (U|W j , r U ), for each j ∈ [p], where, by construction, H P (U|W j ) is a connected component of H P (U). Thus, the property holds at the beginning of iteration l + 1.
In other words, Lemma 8 states that each iteration of BuildNT N (P) deals with a subgraph of H P , whose connected components are in one-to-one correspondence with the valid positions stored in Q. This is illustrated by the next example. In each figure, H Q is shown on the left and the current supertree is shown on the right. For brevity, the figures only exhibit the state of H Q and the supertree after all the nodes at each level are generated. The various valid positions processed by BuildNT N (P) are denoted by U α , for different subscripts α; S α denotes the semi-universal labels in U α , and U ′ α denotes the successor of U α with respect to S α . We write L α as an abbreviation for L(U α ) The root of the tree for L α is r U α and is labeled by S α .

An example
Initially, Q = ((U root , null)). In what follows, the elements of Q are listed from front to rear.
Level 2. Refer to Fig. 6. We have S 11 = {g}, so H P (U ′ 11 ) has two components W 111 and W 112 . Let U 111 = U ′ 11 |W 111 and U 112 = U ′ 11 |W 112 . Then, The only semi-universal labels in U 12 , U 21 , and U 22 are, respectively, e, h, and i. Since none of these labels have proper descendants, each of them is a leaf in the supertree.
After level 2 is processed, Q = ((U 111 , r 11 ), (U 112 , r 11 )). The only semi-universal label in U 112 is d. Since d has no proper descendants, it becomes a leaf in the supertree.
Level 4. Refer to Fig. 8. The only semi-universal labels in U 1111 and U 1112 are, respectively, b and c. Since neither of these labels have proper descendants, each of them is a leaf in the supertree.
After level 4 is processed, Q is empty, and BuildNT N (P) terminates.

Implementation
Here we prove the following result.

Theorem 3
There is an algorithm that, given a profile P of rooted trees, runs in O(M P log 2 M P ) time, and either returns a tree that displays P, if P is compatible, or reports that is P is incompatible otherwise.
We prove this theorem by showing how to implement BuildNT N so that the algorithm runs in O(M P log 2 M P ) on any profile P.
As in the section titled "An iterative version", let H Q denote the subgraph of H P associated with the valid positions in BuildNT N 's queue. By Lemma 8, each valid position U in Q corresponds to one connected component of H Q -namely Desc(U) -and vice-versa. We use this fact in the implementation of BuildNT N : alongside each valid position U in Q, we also store a reference to the respective connected component, together with additional information, described next, to quickly identify semi-universal labels.
Let U be any valid set in Q, let Y = V (H P (U)) be the corresponding connected component of H Q , and let ℓ be any label in Y. Our implementation maintains the following data fields. (Recall that k ℓ is the number of input trees that contain ℓ.). • Y .exposed, a set consisting of all i ∈ [k] such that Y .map(i) = {ℓ} for some ℓ ∈ Y such that ℓ.count = k ℓ . • Y .weight, which equals ℓ∈Y k ℓ . This field is needed for technical reasons, to be explained later.
For the purpose of analysis, we assume that the exposed fields are represented as balanced binary search trees (BSTs), which ensures O(log k) = O(log M P ) time per access and update. The map fields are also implemented using BSTs. We store the set J U = {i ∈ [k] : U (i) � = ∅} as a BST, enabling is to determine in O(log k) time if an index i is in J U , and, if this is the case, to access Y .map(i) . The latter is also stored as a BST, allowing us to search and update Y .
Note that, in practice, hashing may be a better alternative for both exposed and map fields, as it offers expected constant time performance per operation.
The data fields listed above allow us to efficiently retrieve the set S of semi-universal labels in U, as needed in line 5 of BuildNT N (P). Indeed, suppose that U is the valid position extracted from Q at the beginning of an iteration of the while loop of Lines 3-14, and that Y = V (H P (U)). Then, by Lemma 5, we have S = {v ∈ Y .map(i) : i ∈ Y .exposed}. What remains is to devise an efficient way to update these fields for each of the connected components of H P (U) created by replacing U with its successor in Line 11.
Let U ′ be the value of U after Line 11; thus, U ′ is the successor of U. By Observation 2, H P (U ′ ) is obtained from H P (U) through edge and node deletions. We need to (a) Generate the new connected components resulting from these deletions, and (b) Produce the required map, count, and exposed data fields for the various connected components.
We accomplish (a) using the dynamic graph connectivity data structure of Holm et al. [20], which we refer to as HDT. HDT allows us to maintain the list of nodes in each component, as well as the number of these nodes so that, if we start with no edges in a graph with N nodes, the amortized cost of each update is O(log 2 N ). Since H P has O(M P ) nodes, each update takes O(log 2 M P ) time. The total number of edge and node deletions performed by BuildNT N (P) -including all deletions in the interations-is at most the total number of edges and nodes in H P , which is O(M P ). HDT allows us to maintain connectivity information throughout the entire algorithm in O(M P log 2 M P ) time, which is within the time bound claimed in Theorem 3. For part (b), we need to augment HDT in order to maintain the the required data fields for the various connected components created during edge and node deletion. In the next subsections, we describe how to do this. We begin by explaining how to initialize all the required data fields for H P = H P (U root ).

Initializing the data fields
Graph H P (U root ) has a single connected component, Y root = L(P), which is the entire vertex set of the graph. We initialize the data fields as follows.
We initialize the count fields in O(M P ) time as follows: 1. Set ℓ.count to 0 for all ℓ ∈ L(P). Once the count fields are initialized, it is easy to initialize Y root .exposed in O(k) time. Thus, we can initialize all the required fields in O(M P ) time.

Maintaining the data fields
Suppose that all data fields fields are correctly computed for every connected component that is in Q at the beginning of an iteration of the while loop in 3-14 of BuildNT N . We now show how to generate the same fields efficiently for the new connected components created by Line 11.

Computing successor positions
Let U be the valid position extracted from Q at the beginning of an iteration of BuildNT N 's while loop, and let Y = V (Desc(U)) be the associated connected component. Assume all the data fields for Y have been correctly computed. To obtain the successor of U in Line 11 of BuildNT N , we perform the following steps.
1. Identify the set S of semi-universal labels in U. As we saw, this set is given by S = {ℓ ∈ Y .map(i) : i ∈ Y .exposed}. 2. Set Y .map(i) = ∅, for every i ∈ Y .exposed. 3. Make Y .exposed = ∅. 4. For each ℓ ∈ S and each i such that ℓ ∈ L(T i ), do the following.
is a singleton set {α}, increment α.count by one. If α.count = k ℓ , add i to Y .exposed. • Otherwise, Y .map(i) is undefined. 5. For each label ℓ in S, delete the edges incident on ℓ and then ℓ itself, updating the data fields as necessary after each deletion.
The total number of operations on map and exposed fields in Steps 1-4 is O( ℓ∈S k ℓ ). Since each label becomes semi-universal at most once, the total number of operations on map fields over the entire execution of BuildNT N (P) is O( ℓ∈L(P) k ℓ ), which is O(M P ). The same bound holds for updates to count and exposed fields. Next let us focus on how to handle the deletion of a single edge in Step 5.

Deleting an edge
To delete an edge between ℓ and a child α of ℓ, we proceed as follows.
To perform Step 2, we use the well-known technique of scanning the smaller component [21]. We first consult HDT to determine which of Y 1 or Y 2 has fewer nodes. Assume, without loss of generality, that |Y 1 | ≤ |Y 2 |. We initialize Y 1 .weight to 0 and Y 2 .weight to Y .weight. We then scan the labels of Y 1 , incrementing Y 1 .weight by k ℓ for each label ℓ ∈ Y 1 . When the scan of Y 1 is complete, we make Y 2 .weight = Y 2 .weight − Y 1 .weight. We claim that any label ℓ ∈ L(P) is scanned O(log M P ) times over the entire execution of BuildNT N (P). To verify this, let N (ℓ) be the number of nodes in the connected component containing ℓ. Suppose that, initially, N (ℓ) = N . Then, the rth time we scan ℓ, N (ℓ) ≤ N /2 r . Thus, ℓ is scanned O(log N ) times. The claim follows, since N = O(M P ). Therefore, the total number of updates over all labels is O(M P log M P ).
Each execution of Step 6(a) updates each of Y 1 .map(i) and Y 2 .map(i) once.
Step 6(b) is more complex, but can also be accomplished with O(1) data field updates. We omit the (tedious) details. In summary, each execution of step 6 for some β ∈ L(P) performs O(k β ) data field updates.
Let us track the number of data field updates in Step 6 that can be attributed to some specific label β ∈ L(P) over the entire execution of BuildNT N (P). Let w r (β) be the weight of the connected component containing β at the beginning of Step 6, on the rth time that β is considered in that step. Thus, w 0 (β) ≤ ℓ∈L(P) k ℓ . We claim that w r (β) ≤ w 0 (β)/2 r . The reason is that we only consider β if (a) β is contained in one of the two components that result from deleting an edge in step 1 and (b) the component containing β has the smaller weight. Hence, the number of times β is considered in step 6 over the entire execution of BuildNT N (P) is O(log w 0 (β)) , which is O(log M P ), since w 0 (β) = O(M P ). Therefore, the total number of data field updates in Step 6, over all labels in L(P) considered throughout the entire execution of BuildNT N (P), is O(log M P · ℓ∈L(P) k ℓ ), which is O(M P log M P ).

Summary
Let us review the running times of each aspect of our implementation of BuildNT N .
• Initializing the data structures. This has two parts. • Maintaining the data structures. This also has two parts.
• Updating the HDT data structure. There are O(M P ) edge and node deletions, at an amortized cost of O(log 2 M P ) per deletion, yielding a total time of O(M P log 2 M P ). • Maintaining the relevant data fields for the connected components. We have seen that the total number of updates is O(M P log M P ). Assume, conservatively, that each update can be done in O(log M P ) time. Then, this part takes a total of O(M P log 2 M P ) over the entire execution of BuildNT N .
We conclude that the total running time of BuildNT N (P) is O(M P log 2 M P ), completing the proof of Theorem 3.

Discussion
Like our earlier algorithm for compatibility of ordinary phylogenetic trees, the more general algorithm presented here, BuildNT N , is a polylogarithmic factor away from optimality (a trivial lower bound is �(M P ), the time to read the input). BuildNT N has a linear-space implementation, using the results of Thorup [22]. A question to be investigated next is the performance of the algorithm on real data. Another important issue is integrating our algorithm into a synthesis method that deals with incompatible profiles.