 Research
 Open Access
 Published:
Incompatible quartets, triplets, and characters
Algorithms for Molecular Biology volume 8, Article number: 11 (2013)
Abstract
We study a long standing conjecture on the necessary and sufficient conditions for the compatibility of multistate characters: There exists a function f(r) such that, for any set C of rstate characters, C is compatible if and only if every subset of f(r) characters of C is compatible. We show that for every r≥2, there exists an incompatible set C of Ω(r^{2})rstate characters such that every proper subset of C is compatible. This improves the previous lower bound of f(r)≥r given by Meacham (1983), and f(4)≥5 given by Habib and To (2011). For the case when r=3, Lam, Gusfield and Sridhar (2011) recently showed that f(3)=3. We give an independent proof of this result and completely characterize the sets of pairwise compatible 3state characters by a single forbidden intersection pattern.
Our lower bound on f(r) is proven via a result on quartet compatibility that may be of independent interest: For every n≥4, there exists an incompatible set Q of Ω(n^{2}) quartets over n labels such that every proper subset of Q is compatible. We show that such a set of quartets can have size at most 3 when n=5, and at most O(n^{3}) for arbitrary n. We contrast our results on quartets with the case of rooted triplets: For every n≥3, if R is an incompatible set of more than n−1 triplets over n labels, then some proper subset of R is incompatible. We show this bound is tight by exhibiting, for every n≥3, a set of n−1 triplets over n taxa such that R is incompatible, but every proper subset of R is compatible.
Background
The multistate character compatibility (or perfect phylogeny) problem is a basic question in computational phylogenetics [1]. Given a set C of characters, we are asked whether there exists a phylogenetic tree that displays every character in C; if so, C is said to be compatible, and incompatible otherwise. The problem is known to be NPcomplete [2, 3], but certain special cases are known to be polynomiallysolvable [4–10]. See [11] for more on the perfect phylogeny problem.
In this paper we study a long standing conjecture on the necessary and sufficient conditions for the compatibility of multistate characters.
Conjecture 1
There exists a function f(r) such that, for any set C of rstate characters, C is compatible if and only if every subset of f(r) characters of C is compatible.
If Conjecture 1 is true, it would follow that we can determine if any set C of rstate characters is compatible by testing the compatibility of each subset of f(r) characters of C, and, in case of incompatibility, output a subset of at most f(r) characters of C that is incompatible. This would allow us to reduce the character removal problem (i.e., finding a subset of characters to remove from C so that the remaining characters are compatible) to f(r)hitting set which is fixedparameter tractable [12].
A classic result on binary character compatibility shows that f(2)=2; see [1, 6, 13–15]. In 1975, Fitch [16, 17] gave an example of a set C of three 3state characters such that C is incompatible, but every pair of characters in C is compatible; showing that f(3)≥3. In 1983, Meacham [15] generalized this example to rstate characters for every r≥3 demonstrating a lower bound of f(r)≥r for all r; see also [9]. For the case of r=3, Lam, Gusfield, and Sridhar [9] recently established that f(3)=3.
While the previous results could lead one to conjecture that f(r)=r for all r, Habib and To [18] recently disproved this possibility by exhibiting a set C of five 4state characters such that C is incompatible, but every proper subset of the characters in C are compatible, showing that f(4)≥5. They conjectured that f(r)≥r+1 for every r≥4.
The main result of this paper is to prove the conjecture stated in [18] by giving a quadratic lower bound on f(r). Formally, we show that for every r≥2, there exists a set C of rstate characters such that all of the following conditions hold.

1.
C is incompatible.

2.
Every proper subset of C is compatible.

3.
$\leftC\right=\u230a\frac{r}{2}\u230b\xb7\u2308\frac{r}{2}\u2309+1$.
Therefore, $f\left(r\right)\ge \u230a\frac{r}{2}\u230b\xb7\u2308\frac{r}{2}\u2309+1$ for every r≥2.
Our proof relies on a new result on quartet compatibility we believe is of independent interest. We show that for every n≥4, there exists a set Q of quartets over a set of n labels such that all of the following conditions hold.

1.
Q is incompatible.

2.
Every proper subset of Q is compatible.

3.
$\leftQ\right=\u230a\frac{n2}{2}\u230b\xb7\u2308\frac{n2}{2}\u2309+1$.
This is an improvement over the previous lower bound on the maximum cardinality of such an incompatible set of quartets of n−2 given in [3]. We show that such a set of quartets can have size at most 3 when n=5, and at most O(n^{3}) for arbitrary n. We note here that the construction given in [18] showing that f(4)≥5 can be viewed as a special case of the construction given here when n=6.
We study the compatibility of threestate characters further. The work of [9] completely characterized the sets of pairwise compatible 3state characters by the existence of one of four forbidden intersection patterns. An alternative characterization of this result was given in [10] and was partially derived using the results of [9]. In this paper, we give a proof that f(3)=3 that is independent of the results in [9], and we completely characterize the sets of pairwise compatible 3state characters by a single forbidden intersection pattern.
We contrast our result on quartet compatibility with a result on the compatibility of rooted triplets: For every n≥3, if R is an incompatible set of triplets over n labels, and R>n−1, then some proper subset of R is incompatible. We show this bound is tight by exhibiting, for every n≥3, a set of n−1 triplets over n labels such that R is incompatible, but every proper subset of R is compatible.
Preliminaries
Given a graph G, we represent the vertices and edges of G by V(G) and E(G) respectively. We use the abbreviated notation uv for an edge {u,v}∈E(G). For any e∈E(G), G−e represents the graph obtained from G by deleting edge e. For an integer i, we use [i] to represent the set {1,2,⋯,i}.
Unrooted phylogenetic trees
An unrooted phylogenetic tree (or just tree) is a tree T whose leaves are in one to one correspondence with a label set L(T), and has no vertex of degree two. See Figure 1(a) for an example. For a collection $\mathcal{T}$ of trees, the label set of $\mathcal{T}$, denoted $L\left(\mathcal{T}\right)$, is the union of the label sets of the trees in $\mathcal{T}$. A tree is binary if every internal (nonleaf) vertex has degree three. A quartet is a binary tree with exactly four leaves. A quartet with label set {a,b,c,d} is denoted a bc d if the path between the leaves labeled a and b does not intersect with the path between the leaves labeled c and d.
For a tree T, and a label set L⊆L(T), the restriction of T to L, denoted by TL, is the tree obtained from the minimal subtree of T connecting all the leaves with labels in L by suppressing vertices of degree two. See Figure 1(b) for an example. A tree T displays another tree T^{′}, if T^{′} can be obtained from TL(T^{′}) by contracting edges. A tree T displays a collection of trees $\mathcal{T}$ if T displays every tree in $\mathcal{T}$. If such a tree T exists, then we say that $\mathcal{T}$ is compatible; otherwise, we say that $\mathcal{T}$ is incompatible. See Figure 1(a) for an example. Determining if a collection of unrooted trees is compatible is NPcomplete [3].
Multistate characters
There is also a notion of compatibility for sets of partitions of a label set L. A character χ on L is a partition of L; the parts of χ are called states. If χ has at most r parts, then χ is an rstate character. Given a tree T with L=L(T) and a state s of χ, we denote by T_{ s }(χ) the minimal subtree of T connecting all leaves with labels having state s for χ. We say that χ is convex on T, or equivalently T displays χ, if the subtrees T_{ i }(χ) and T_{ j }(χ) are vertex disjoint for all states i and j of χ where i≠j. A collection C of characters is compatible if there exists a tree T on which every character in C is convex. If no such tree exists, then we say that C is incompatible. See Figure 1(a) for an example. The perfect phylogeny problem (or character compatibility problem) is to determine whether a given set of characters is compatible.
For a collection C of characters, the intersection graph of C which we will denote by G(C), is the undirected graph G=(V,E) which has a vertex c_{ i } for each character c∈C and each state i of c, and an edge c_{ i }d_{ j } precisely when there is a taxon having state i for character c and state j for character d. Note that G(C) cannot have an edge between vertices associated with different states of the same character.
A graph G is chordal if there are no induced chordless cycles of length four or greater in H. In [19], Buneman established a fundamental connection between the perfect phylogeny problem and chordal graphs which we now describe. For a given set C of characters, suppose we color each of the vertices of G(C) by assigning a unique color to each character c∈C, and giving each vertex of G(C) corresponding to a state of c with the color assigned to the character c. A proper triangulation of G(C) is a chordal supergraph of G(C) such that every edge has endpoints with different colors.
Theorem 1.
A set C of characters is compatible if and only if G(C) has a proper triangulation.
Since there is no proper triangulation for a cycle in G(C) involving only vertices from two characters, we have the following corollary.
Corollary 1.
Let C be a collection of two characters. Then C is compatible if and only if G(C) is acyclic.
Quartet rules
We now introduce quartet (closure) rules which were originally used in the contexts of psychology [20] and linguistics [21]. The idea is that for a collection Q of quartets, any tree that displays Q may also necessarily display another quartet q∉Q, and if so we write Q⊩q.
Example 1.
Let Q={a bc e,a ec d}. Then the tree of Figure 1(b) displays Q, and furthermore, it is easy to see that it is the only tree that displays Q. Hence, Q⊩a bd e, Q⊩a bc d, and Q⊩b ec d.
We use the following quartet rules in this paper:
For the purposes of this paper, we define the closure of an arbitrary collection Q of quartets, denoted Q^{∗}, as the minimal set of quartets that contains Q, and has the property that if for some q_{1},q_{2}∈Q^{∗}, {q_{1},q_{2}}⊩q_{3} using either (R1) or (R2), then q_{3}∈Q^{∗}. Clearly, any tree that displays Q must also display Q^{∗}. We will use the following lemma which follows by repeated application of (R!) and is formally proven in [22].
Lemma 1.
Let Q be an arbitrary set of quartets with {x,y,z_{1},…,z_{ k }}⊆L(Q). If
then x yz_{1}z_{ k }∈Q^{∗}.
Incompatible quartets
For every s,t≥2, we fix a set of labels L_{s,t}={a_{1},a_{2},…,a_{ s },b_{1},b_{2},…,b_{ t }} and define the set
of quartets with L(Q_{s,t})=L_{s,t}. We denote the quartet a_{1}b_{1}a_{ s }b_{ t } by q_{0}, and a quartet of the form a_{ i }a_{i+1}b_{ j }b_{j+1} by q_{i,j}.
Observation 1.
For all s,t ≥ 2, Q_{s,t}=(s−1)(t−1) + 1.
Lemma 2.
For all s,t≥2, Q_{s,t} is incompatible.
Proof.
For each i∈[s−1],
Then, by Lemma 1, it follows that for each i∈[s−1], ${a}_{i}{a}_{i+1}{b}_{1}{b}_{t}\in {Q}_{s,t}^{\ast}$. So,
Then, again by Lemma 1, it follows that ${b}_{1}{b}_{t}{a}_{1}{a}_{s}\in {Q}_{s,t}^{\ast}$. But then $\left\{{a}_{1}{b}_{1}\right{a}_{s}{b}_{t},{b}_{1}{b}_{t}\left{a}_{1}{a}_{s}\right\}\subseteq {Q}_{s,t}^{\ast}$. It follows that any tree that displays Q_{s,t} must display both a_{1}b_{1}a_{ s }b_{ t } and b_{1}b_{ t }a_{1}a_{ s }. However, no such tree exists. Hence, Q_{s,t} is incompatible. □
Lemma 3.
For all s,t≥2, every proper subset of Q_{s,t} is compatible.
Proof.
Since every subset of a compatible set of quartets is compatible, it suffices to show that for every q∈Q_{s,t}, Q_{s,t}∖{q} is compatible. Let q∈Q_{s,t}. Either q=q_{0} or q=q_{x,y} for some 1≤x<s and 1≤y<t. In either case, we exhibit a tree witnessing that Q_{s,t}∖{q} is compatible. □

Case 1. Suppose q=q_{0}. We build the tree T as follows: There is a node ℓ for each label ℓ∈L_{s,t} and two additional nodes a and b along with the edge ab. There is an edge a_{ x }a for every a_{ x }∈L_{s,t}, and an edge b_{ x }b for every b_{ x }∈L_{s,t}. There are no other nodes or edges in T. See Figure 2(a) for an illustration. Now consider any quartet q∈Q_{s,t}∖{q_{0}}. Then q=a_{ i }a_{i+1}b_{ j }b_{j+1} for some 1≤i<s and 1≤j<t. Then, the minimal subgraph of T connecting leaves with labels in {a_{ i },a_{i+1},b_{ j },b_{j+1}} is the quartet q. Hence T displays q.

Case 2. Suppose q=q_{x,y} for some 1≤x<s and 1≤y<t. We build the tree T as follows: There is a node ℓ for each label ℓ∈L_{s,t} and six additional nodes a_{ ℓ }, b_{ ℓ }, ℓ, h, a_{ h }, and b_{ h }. There are edges a_{ ℓ }ℓ, b_{ ℓ }ℓ, ℓ h, h a_{ h }, and h b_{ h }. For every a_{ i }∈L_{s,t}, there is an edge a_{ i }a_{ ℓ } if i≤x, and an edge a_{ i }a_{ h } if i>x. For every b_{ j }∈L_{s,t} there is an edge b_{ j }b_{ ℓ } if j≤x, and an edge b_{ j }b_{ h } if j>y. There are no other nodes or edges in T. See Figure 2(b). Now consider any quartet q∈Q_{s,t}∖{q_{x,y}}. Either q=q_{0} or q=q_{i,j} where i≠x or j≠y. If q=q_{0}, then the minimal subgraph of T connecting leaves with labels in {a_{1},b_{1},a_{ s },b_{ t }} is the subtree of T induced by the nodes in {a_{1},a_{ ℓ },ℓ,b_{ ℓ },b_{1},a_{ s },a_{ h },h,b_{ h },b_{ t }}. Suppressing all degree two vertices results in a tree that is the same as q_{0}. So T displays q. So assume that q=a_{ i }a_{i+1}b_{ j }b_{j+1} where i≠x or j≠y. We define the following subset of the nodes in T:
$$\phantom{\rule{15.0pt}{0ex}}V\phantom{\rule{0.3em}{0ex}}=\phantom{\rule{0.3em}{0ex}}\left\{\begin{array}{c}\phantom{\rule{0.3em}{0ex}}\{{a}_{i},{a}_{i+1},{a}_{\ell},\ell ,{b}_{\ell},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i<x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j<y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{i},{a}_{i+1},{a}_{\ell},\ell ,{b}_{y},{b}_{\ell},h,{b}_{h},{b}_{y+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i<x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j=y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{i},{a}_{i+1},{a}_{\ell},\ell ,h,{b}_{h},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i<x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j>y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{x},{a}_{\ell},\ell ,h,{a}_{h},{a}_{x+1},{b}_{\ell},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i=x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j<y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{x},{a}_{\ell},\ell ,h,{a}_{h},{a}_{x+1},{b}_{h},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i=x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j>y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{j},{a}_{j+1},{a}_{h},h,\ell ,{b}_{\ell},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i>x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j<y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{j},{a}_{j+1},{a}_{h},h,{b}_{y},{b}_{\ell},\ell ,{b}_{h},{b}_{y+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i>x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j=y,\\ \phantom{\rule{0.3em}{0ex}}\{{a}_{j},{a}_{j+1},{a}_{h},h,{b}_{h},{b}_{j},{b}_{j+1}\}\phantom{\rule{2em}{0ex}}\phantom{\rule{2em}{0ex}}\phantom{\rule{0.3em}{0ex}}\text{if}\phantom{\rule{0.3em}{0ex}}i>x\phantom{\rule{0.3em}{0ex}}\text{and}\phantom{\rule{0.3em}{0ex}}j>\mathrm{y.}\end{array}\right.$$
Now, the subgraph of T induced by the nodes in V is the minimal subgraph of T connecting leaves with labels in q. Suppressing all degree two vertices gives q. Hence, T displays q.
With $s=\u230a\frac{n}{2}\u230b$ and $t=\u2308\frac{n}{2}\u2309$, Observation 1and Lemmas 2 and 3 imply the following theorem.
Theorem 2.
For every integer n≥4, there exists a set Q of quartets over n taxa such that all of the following conditions hold.

1.
Q is incompatible.

2.
Every proper subset of Q is compatible.

3.
$\leftQ\right=\u230a\frac{n2}{2}\u230b\xb7\u2308\frac{n2}{2}\u2309+1$.
Incompatible quartets on five taxa
When Q is a set of quartets over five taxa, we show that the set of quartets given by Theorem 2 is as large as possible. We hope that the technique used in the proof of the following theorem might be useful in proving tight bounds for n>5.
Theorem 3.
If Q is an incompatible set of quartets over five taxa such that every proper subset of Q is compatible, then Q≤3.
Lemma 4.
Let Q be an incompatible set of quartets with L(Q)={a,b,c,d,e} and q_{0}=a bc d∈Q. We will show that Q contains an incompatible subset of at most three quartets. If Q contains two different quartets on the same four taxa, then Q must contain an incompatible pair of quartets. So, we may assume that each quartet is on a unique subset of four of the five taxa. Hence, every pair of quartets in Q shares three taxa in common. We have the following two cases.

Case 1: Q contains at least one of the quartets a cb e, a cd e, a db e, a dc e, a eb c, a eb d, b cd e, or b dc e. W.l.o.g. we may assume that Q contains q_{1}=a cd e, as all other cases are symmetric. By (R2), {q_{0},q_{1}}⊩a bc e. Then, by (R1), {q_{0},q_{1},a bc e}⊩a bd e. Then, again by (R1), {q_{0},q_{1},a bc e,a bd e}⊩b cd e. Now let Q^{′}={q_{0},q_{1},a bc e,a bd e,b cd e}. Now, any quartet in Q must be either in Q^{′} or be pairwise incompatible with a quartet in Q^{′}. Since Q^{′} is compatible, but by assumption, Q is incompatible, Q must contain a quartet q_{2} that is pairwise incompatible with some quartet in Q^{′}. Hence, {q_{0},q_{1},q_{2}} is an incompatible subset of Q.

Case 2: Q contains none of the quartets a cb e, a cd e, a db e, a dc e, a eb c, a eb d, b cd e, or b dc e. Then every quartet in Q is either of the form a bx y where {x,y}≠{c,d}, or c dx y where {x,y}≠{a,b}. But then Q is compatible, contradicting our assumption that Q is incompatible.
In either case, the theorem holds. □
Incompatible quartets on arbitrarily many taxa
We say a set Q of compatible quartets is redundant if for some q∈Q, Q∖{q}⊩q; otherwise, we say that Q is irredundant. The following lemma establishes a connection between sets of irredundant quartets and minimal sets of incompatible quartets.
Lemma 4.
If Q is incompatible, but every proper subset of Q is compatible, then every proper subset of Q is irredundant.
Proof.
Suppose that Q is incompatible and every proper subset of Q is compatible. Furthermore, suppose that some proper subset Q^{′} of Q is redundant. Since every compatible superset of a redundant set of quartets is also redundant, we may assume w.l.o.g., that there is a unique quartet q∈Q∖Q^{′} (i.e., Q=Q^{′}+1). Since Q^{′} is redundant, there exists a q^{′}∈Q^{′} such that Q^{′}∖{q^{′}}⊩q^{′}. But then (Q^{′}∖{q^{′}})∪{q} is incompatible, contradicting that every proper subset of Q is compatible. □
It follows from Lemma 4 that any upper bound on the maximum cardinality of an irredundant set of quartets can be used to place an upper bound on the maximum cardinality of a set of quartets satisfying the first two conditions of Theorem 2. The theorem follows from [22].
Theorem 4.
Let Q be a set of quartets over a set of n taxa. If Q is irredundant, then Q has cardinality at most (n−3)(n−2)^{2}/3.
Lemma 4 together with Theorem 4 gives the following upper bound on the maximum cardinality of a set Q of quartets over n>5 taxa that satisfies the first two conditions of Theorem 2.
Theorem 5.
Let Q be a set of incompatible quartets over a set of n taxa such that every proper subset of Q is compatible. Then Q≤(n−3)(n−2)^{2}/3+1.
Incompatible characters
There is a natural correspondence between quartet compatibility and character compatibility that we now describe. Let Q be a set of quartets, n=L(Q), and r=n−2. For each q=a bc d∈Q, we define the rstate character corresponding to q, denoted χ_{ q }, as the character where a and b have state 0 for χ_{ q }; c and d have state 1 for χ_{ q }; and, for each ℓ∈L(Q)∖{a,b,c,d}, there is a state s of χ_{ q } such that ℓ is the only label with state s for character χ_{ q } (see Example 2). We define the set of rstate characters corresponding to Q by
Example 2.
Consider the quartets and characters given in Figure 1(a): ${\chi}_{{q}_{1}}$ is the character corresponding to q_{1}, ${\chi}_{{q}_{2}}$ is the character corresponding to q_{2}, and ${\chi}_{{q}_{3}}$ is the character corresponding to q_{3}.
The following lemma relating quartet compatibility to character compatibility is well known [24], and its proof is omitted here.
Lemma 5.
A set Q of quartets is compatible if and only if C_{ Q } is compatible.
The next theorem allows us to use our result on quartet compatibility to establish a lower bound on f(r).
Theorem 6.
Let Q be a set of incompatible quartets over n labels such that every proper subset of Q is compatible, and let r=n−2. Then, there exists a set C of Qrstate characters such that C is incompatible, but every proper subset of C is compatible.
Proof.
We claim that C_{ Q } is such a set of incompatible rstate characters. Since for two quartets q_{1},q_{2}∈Q, ${\chi}_{{q}_{1}}\ne {\chi}_{{q}_{2}}$, it follows that C_{ Q }=Q. Since Q is incompatible, it follows by Lemma 5 that C_{ Q } is incompatible. Let C^{′} be any proper subset of C. Then, there is a proper subset Q^{′} of Q such that ${C}^{\prime}={C}_{{Q}^{\prime}}$. Then, since Q^{′} is compatible, it follows by Lemma 5 that C^{′} is compatible. □
Theorem 2 together with Theorem 6 gives the main theorem of this paper.
Theorem 7.
For every integer r≥2, there exists a set C of rstate characters such that all of the following hold.

1.
C is incompatible.

2.
Every proper subset of C is compatible.

3.
$\leftC\right=\u230a\frac{r}{2}\u230b\xb7\u2308\frac{r}{2}\u2309+1$.
Proof.
By Theorem 2 and Observation 1, there exists a set Q of $\u230a\frac{r}{2}\u230b\xb7\u2308\frac{r}{2}\u2309+1$ quartets over r+2 labels that that are incompatible, but every proper subset is compatible, namely ${Q}_{\u230a\frac{r+2}{2}\u230b,\u2308\frac{r+2}{2}\u2309}$. The theorem follows from Theorem 6. □
The quadratic lower bound on f(r) follows from Theorem 7.
Corollary 2
.
ThreeState Characters
In the remainder of this section we focus on the case when r=3, and thus, fix C to be an arbitrary set of 3state characters over a set S of taxa. Lam, Gusfield, and Sridhar [9] recently established that f(3)=3, and they completely characterized the sets of pairwise compatible 3state characters by the existence of one of four forbidden intersection patterns. We give an independent proof that f(3)=3. We then completely characterize the sets of pairwise compatible 3state characters by a single forbidden intersection pattern. Our proof uses several structural results from the algorithm for the threestate perfect phylogeny problem given by Kannan and Warnow [7].
The Algorithm of Kannan and Warnow
The algorithm of [7] takes a divide and conquer approach to determining the compatibility of a set of threestate characters. An instance is reduced to subproblems by finding a partition S_{1},S_{2} of the taxon set S of C with both of the following properties:

1.
2≤S _{ i }≤n−2,i=1,2.

2.
Whenever C is compatible S there is a perfect phylogeny P that contains an edge e whose removal breaks P into subtrees P _{1} and P _{2} with L(P _{ i })=S _{ i },i=1,2.
A partition of S satisfying both of these properties is a legal partition, and the following theorem shows that finding such a partition for a given set of characters is the crux of the algorithm.
Theorem 8.
[7] Given a set C of three state characters, we can in O(n k) time either find a legal partition of S of determine that the set of characters is incompatible.
Finding a legal partition
We now discuss the manner in which such a legal partition is found for a set of threestate characters C. Let T be a tree witnessing that C is compatible. The canonical labeling of T is the labeling where, for each internal node v of T, and each character α∈C, if there are leaves x and y in different components of T−{v} such that α(x)=α(y), then α(v)=α(x); otherwise α(v)=∗ where ∗ denotes a dummy state for C. Note that such a labeling of T always exists and is unique. We will assume that every compatible tree for C is canonically labeled.
The treestructure for a character α in T is formed by repeatedly contracting edges of T connecting nodes that have the same state (other than ∗) for α. Note that this tree does not depend on the sequence of edgecontractions and is thus well defined. Furthermore, there is exactly one node for each state (other than the dummy state) of α, and each node labeled by ∗ has degree at least three. A treestructure for α that is formed from some compatible tree for C is called a realizable treestructure for α. There are four possible realizable treestructures for a threestate character α which are shown in Figure 3.
To find a realizable tree structure for a character α, the algorithm examines the pairwise intersection patterns of α with every other character β∈∈cC, and applies the following rules to rule out possible tree structures for α.
Rule 1.
Let α and β be two characters of C. If, under some relabeling of the states of α and β, we have that α_{1}⊆β_{1}, α_{2}∩β_{2}≠∅, and α_{3}∩β_{2}≠∅, then P^{1} is not a realizable treestructure for α. If this is the case, we say that α and β match Rule 1 with respect to α_{1}.
Rule 2.
^{a}Let α and β be two characters of C. If, under some relabeling of the states of α and β, we have that α_{1}∩β_{1}≠∅, α_{2}∩β_{1}≠∅, α_{2}∩β_{2}≠∅, and α_{3}∩β_{2}≠∅, then P^{2} is the only possible realizable treestructure for α. If this is the case, we say that α and β match Rule 2 with respect to α_{2}.
The set ${Q}_{\alpha}^{C}$ of candidate treestructures for α are all of those possible treestructures for α that are not ruled out after comparing the intersection pattern of α with every other character in C and applying Rules 1 and 2.
The following theorem which follows from [7] shows that a legal partition is found by choosing an arbitrary α∈C for which ${Q}_{\alpha}^{C}\ne \varnothing $. Furthermore, if there is an α∈C for which ${Q}_{\alpha}^{C}=\varnothing $, then C is incompatible.
Theorem 9
([7]).If ${Q}_{\alpha}^{C}\ne \varnothing $, then we can find a legal partition of S.
Corollary 3.
A set C of 3state characters is compatible if and only if ${Q}_{\alpha}^{C}\ne \varnothing $ for every α∈C.
Tight bounds on threestate character compatibility
We use Corollary 3 to give upper bounds on the maximum cardinality of a minimal set of incompatible threestate characters.
Theorem 10.
Let C be a set of threestate characters on species set S. Then C is incompatible if and only if there exists a character α∈C, and two distinct states α_{ i } and α_{ j } of α, such that both of the following hold:

1.
There is a β∈∈cC where the intersection pattern of α and β matches Rule 2 with respect to α _{ i }.

2.
There is a γ∈C where the intersection pattern of α and γ matches Rule 2 with respect to α _{ j }.
Proof.
(⇒) If C is pairwise incompatible, then by Corollary 1, there is a pair α,β ∈ C whose intersection graph contains a cycle. Since the intersection graph is bipartite, this cycle must have length at least four and contain at least two states of each character. Let α_{ i } and α_{ j } be the two states of α on this cycle. Then, the intersection pattern of α and β matches Rule 2 with respect to both α_{ i } and α_{ j }, and so the theorem holds. So we may assume that C is incompatible but pairwise compatible.
It follows from Corollary 3 that there exists an α∈C such that ${Q}_{\alpha}^{C}=\varnothing $. Then there must exist a character β∈∈cC such that the intersection pattern of α and β matches Rule 2 with respect to some state α_{ i } of α; otherwise ${S}^{\ast}\in {Q}_{\alpha}^{C}$. Hence, ${Q}_{\alpha}^{C}\subseteq \left\{{P}^{i}\right\}$. Then, since ${Q}_{\alpha}^{C}=\varnothing $, there must be a character γ∈C such that the intersection pattern of α and γ places a constraint on ${Q}_{\alpha}^{C}$ that prevents ${Q}_{\alpha}^{C}$ from containing P^{i}. There are two possibilities.
Case 1: There is a state α_{ j } of α where j≠i and the intersection pattern of α and γ matches Rule 2 with respect to α_{ j }. In this case the theorem holds.
Case 2: The intersection pattern of α and γ matches Rule 1 with respect to α_{ i }. W.l.o.g., we fix i=1, and relabel the states of α, β, and γ so that α_{1}∩β_{1}≠∅, α_{1}∩β_{2}≠∅, α_{2}∩β_{1}≠∅, α_{3}∩β_{2}≠∅, α_{1}⊆γ_{1}, α_{2}∩γ_{2}≠∅, and α_{3}∩γ_{2}≠∅. Such a labeling exists since, by assumption, α and β matches Rule 2 with respect to α_{1}, and α and γ matches Rule 1 with respect to α_{1}.
If α_{2}∩γ_{1}≠∅, then the intersection pattern of α and γ matches Rule 2 with respect to α_{2}, in which case the theorem holds. If α_{3}∩γ_{1}≠∅, then the intersection pattern of α and γ matches Rule 2 with respect to α_{3}, in which case the theorem holds. So we may assume hat α_{1}=γ_{1}. Now, since α_{1}∩β_{1}≠∅, α_{1}∩β_{2}≠∅, and α_{1}=γ_{1}, we have that both β_{1}∩γ_{1}≠∅ and β_{2}∩γ_{2}≠∅.
γ_{3} must have a nonempty intersection with at least one state of α, and since α_{1}=γ_{1}, we have that α_{1}∩γ_{3}=∅. So γ_{3} has a nonempty intersection with either α_{2} or α_{3}. Due to the symmetry of the intersection graph of α and β, we may assume, w.l.o.g., that α_{3}∩γ_{3}≠∅.
By assumption, α_{2}∩γ_{1}=∅, and if α_{2}∩γ_{3}≠∅, then the intersection graph of α and β contains a cycle, contradicting our assumption that C is pairwise compatible. So we may assume that α_{2}⊂γ_{2}. Then, since β_{1}∩α_{2}≠∅, we have that β_{1}∩γ_{2}≠∅.
Let s∈α_{3}∩β_{2}. Since, by assumption, α_{3}∩γ_{1}=∅, we have that either s∈γ_{2} or s∈γ_{3}. However, if s∈γ_{2}, then β_{2}∩γ_{2}≠∅ and intersection graph of β and γ contains a cycle, contradicting our assumption that C is pairwise compatible. Hence s∈γ_{3} and β_{2}∩γ_{3}≠∅.
We have now established all of the edges of the intersection graph of α, β, and γ represented by the solid edges in Figure 4. Now, let s_{5}∈α_{3}∩γ_{2}. Now s_{5} must be in some state of β. If s_{5}∈β_{1}, then s_{5}∈β_{1}∩α_{3} and the intersection graph of β and α contains a cycle, contradicting our assumption that C is pairwise compatible. If s_{5}∈β_{2}, then s_{5}∈β_{2}∩γ_{2}, and the intersection graph of β and γ contains a cycle, again contradicting our assumption that C is pairwise compatible. Hence s_{5}∈β_{3}. Then, we have that s_{5}∈β_{3}∩α_{3} and s_{5}∈β_{3}∩γ_{2}, witnessing the dotted edges in Figure 4. So we have that the intersection pattern of β and α matches Rule 2 with β_{2} as witness, and the intersection pattern of β and γ matches Rule 2 with β_{1} as witness. Hence the theorem holds. □
Note that in the statement of Theorem 10, the characters β and γ are not necessarily distinct. In cases where they are not distinct, C contains an incompatible pair.
Corollary 4.
A set C of 3state characters is compatible if and only if every subset of at most three characters of C is compatible.
In [9], it was also shown that we can determine the compatibility of a pairwise compatible set C of threestate characters by testing the intersection patterns of C for the existence of one of a set of four forbidden patterns. As a corollary to Theorem 10, we have that a single forbidden pattern suffices to determine the compatibility of C.
Corollary 5.
A pairwise compatible set C of 3state characters is compatible if and only if the partition intersection graph of C does not contain, up to relabeling of characters and states, the subgraph of Figure 5.
Note that each edge of the graph of Figure 5 has one endpoint which is a state in α. It follows that we can find such a subgraph in the partition intersection graph of C by testing the intersection pattern of each pair of characters in C[10]. Furthermore, all p occurrences of the forbidden subgraph in the intersection graph of m characters on n taxa can be found in O(m^{2}n+p) time. Whereas the forbidden subgraph given here is witnessed by eight taxa (or edges), each of the four forbidden subgraphs of [9] are witnessed by five taxa, making them better suited for taxon removal problems.
Incompatible Triplets
A rooted phylogenetic tree (or just rooted tree) is a tree whose leaves are in one to one correspondence with a label set L(T), has a distinguished vertex called the root, and no vertex other than the root has degree two. See Figure 6(a) for an example. A rooted tree is binary if the root vertex has degree two, and every other internal (nonleaf) vertex has degree three. A triplet is a rooted binary tree with exactly three leaves. A triplet with label set {a,b,c} is denoted a bc if the path between the leaves labeled a and b avoids the path between the leaf labeled c and the root vertex. For a tree T, and a label set L⊆L(T), let T^{′} be the minimal subtree of T connecting all the leaves with labels in L. The restriction of T to L, denoted by TL, is the rooted tree obtained from T^{′} by distinguishing the vertex closest to the root of T as the root of T^{′}, and suppressing every vertex other than the root having degree two. A rooted tree T displays another rooted tree T^{′} if T^{′} can be obtained from TL(T^{′}) by contracting edges. A rooted tree T displays a collection of rooted trees $\mathcal{T}$ if T displays every tree in $\mathcal{T}$. If such a tree T exists, then we say that $\mathcal{T}$ is compatible; otherwise, we say that $\mathcal{T}$ is incompatible. Given a collection of rooted trees $\mathcal{T}$, it can be determined in polynomial time if $\mathcal{T}$ is compatible [3],[25].
The following theorems follow from the connection between collections of unrooted trees with at least one common label across all the trees, and collections of rooted trees [3].
Theorem 11.
Let Q be a collection of quartets where every quartet in Q shares a common label ℓ. Let R be the set of triplets such that there exists a triplet a bc in R if and only if there exists a quartet a bc ℓ in Q. Then, Q is compatible if and only if R is compatible.
Let R be a collection of triplets. For a subset S⊆L(R), we define the graph [R,S] as the graph having a vertex for each label in S, and an edge {a,b} if and only if a bc∈R for some c∈S. The following theorem is from page 439 of [26].
Theorem 12.
A collection R of rooted triplets is compatible if and only if [R,S] is not connected for every S⊆L(R) with S≥3.
Corollary 6.
Let R be a set of rooted triplets such that R is incompatible but every proper subset of R is compatible. Then, [R,L(R)] is connected.
We now contrast our result on quartet compatibility with a result on triplets.
Theorem 13.
For every n≥3, if R is an incompatible set of triplets over n labels, and R>n−1, then some proper subset of R is incompatible.
Proof.
For sake of contradiction, let R be a set of triplets such that R is incompatible, every proper subset of R is compatible, L(R)=n, and R>n−1. The graph [R,L(R)] will contain n vertices and at least n edges. Since each triplet in R is distinct, there will be a cycle C of length at least three in [R,L(R)]. Since R is incompatible but every proper subset of R is compatible, by Corollary 6, [R,L(R)] is connected.
Consider any edge e in the cycle C. Let t be the triplet that contributed edge e in [R,L(R)]. Let R^{′}= R∖t. Since the graph [R,L(R)]−e is connected, [R^{′},L(R^{′})] is connected. By Theorem 12, R^{′} is incompatible. But R^{′}⊂ R, contradicting that every proper subset of R is compatible. □
To show the bound is tight, we first prove a more restricted form of Theorem 2.
Theorem 14.
For every n≥4, there exists a set of quartets Q with L(Q)=n, and a label ℓ∈L(Q), such that all of the following hold.

1.
Every q∈Q contains a leaf labeled by ℓ.

2.
Q is incompatible.

3.
Every proper subset of Q is compatible.

4.
Q=n−2.
Proof.
Consider the set of quartets Q_{2,n−2}. From Lemmas 2 and 3, Q_{2,n−2} is incompatible but every proper subset of Q_{2,n−2} is compatible. The set Q_{2,n−2} contains exactly n−2 quartets. From the construction, there are two labels in L which are present in all the quartets in Q_{2,n−2}. Set one of them to be ℓ. □
The following is a consequence of Theorems 14 and 11.
Corollary 7.
For every n≥3, there exists a set R of triplets with L(R)=n such that all of the following hold.

1.
R is incompatible.

2.
Every proper subset of R is compatible.

3.
R=n−1.
The generalization of the FitchMeacham examples given in [9] can also be expressed in terms of triplets. For any r≥2, let L={a,b_{ 1 },b_{ 2 },⋯,b_{ r }}. Let
Let Q={a bc ℓ:a bc∈R_{ r }} for some label ℓ∉L. The set C_{ Q } of rstate characters corresponding to the quartet set Q is exactly the set of characters built for r in [9]. In the partition intersection graph of C_{ Q }, (following the terminology in [9]) labels ℓ and a correspond to the end cliques and the rest of the r labels {b_{ 1 },b_{ 2 },⋯,b_{ r }} correspond to the r tower cliques. From Lemma 5 and Theorem 11, R_{ r } is compatible if and only of Q is compatible.
Conclusion
We have shown that for every r≥2, $f\left(r\right)\ge \u230a\frac{r}{2}\u230b\xb7\u2308\frac{r}{2}\u2309+1$, by showing that for every n≥4, there exists an incompatible set Q of $\u230a\frac{n2}{2}\u230b\xb7\u2308\frac{n2}{2}\u2309+1$ quartets over a set of n labels such that every proper subset of Q is compatible. Previous results [1],[6],[9],[13][15], along with our discussion in Section Incompatible Characters, show that our lower bound on f(r) is tight for r=2 and r=3. For quartets, our discussion in Section Incompatible quartets gives an upper bound on the maximum cardinality of a minimal set of incompatible quartets. However, this argument does not extend to multistate characters. Indeed, an upper bound on the maximum cardinality of a minimal set of incompatible rstate characters remains a central open question. We give the following conjecture.
Conjecture 2.
f(r)∈Θ(r^{2}).
A less ambituous goal would be to narrow the gap between the upper bound of O(n^{3}) and lower bound of Ω(n^{2}) on the maximum cardinality of a minimal incompatible set of quartets over n taxa given in Section Incompatible Quartets. Note that, due to Theorem 6, a proof of Conjecture 2 would also show that the number of incompatible quartets given in the statement of Theorem 2 is also as large as possible.
Endnote
^{a}Rule 2 was state incorrectly in[7].
References
 1.
Semple C, Steel M: Phylogenetics. Oxford Lecture Series in Mathematics and its Applications. USA: Oxford University Press 2003.
 2.
Bodlaender H, Fellows M, Warnow T: Two strikes against perfect phylogeny. Automata, Languages and Programming, Volume 623 of Lecture Notes in Computer Science. Edited by: Kuich W. Berlin/Heidelberg: Springer 1992, 273283.
 3.
Steel M: The complexity of reconstructing trees from qualitative characters and subtrees. J Classif. 1992, 9: 91116. 10.1007/BF02618470.
 4.
Agarwala R, FernándezBaca D: A polynomialtime algorithm for the perfect phylogeny problem when the number of character states is fixed. SIAM J Comput. 1994, 23 (6): 12161224. 10.1137/S0097539793244587.
 5.
Dress A, Steel M: Convex tree realizations of partitions. Appl Math Lett. 1992, 5 (3): 36. 10.1016/08939659(92)900266.
 6.
Gusfield D: Efficient algorithms for inferring evolutionary trees. Networks. 1991, 21: 1928. 10.1002/net.3230210104.
 7.
Kannan S, Warnow T: Inferring evolutionary history from DNA sequences. SIAM J Comput. 1994, 23 (4): 713737. 10.1137/S0097539791222171.
 8.
Kannan S, Warnow T: A fast algorithm for the computation and enumeration of perfect phylogenies. SIAM J Comput. 1997, 26 (6): 17491763. 10.1137/S0097539794279067.
 9.
Lam F, Gusfield D, Sridhar S: Generalizing the splits equivalence theorem and four Gamete condition: perfect phylogeny on threestate characters. SIAM J Discrete Math. 2011, 25 (3): 11441175. 10.1137/090776305.
 10.
Shutters B, FernándezBaca D: A simple characterization of the minimal obstruction sets for threestate perfect phylogenies. Appl Math Lett. 2012, 25 (9): 12261229. 10.1016/j.aml.2012.02.060.
 11.
FernándezBaca D: The Perfect Phylogeny Problem. Steiner Trees in Industry. Dordrecht: Kluwer 2001, 203234.
 12.
Niedermeier R, Rossmanith P: An efficient fixedparameter algorithm for 3Hitting Set. J Discrete Algorithms. 2003, 1: 89102. 10.1016/S15708667(03)000091.
 13.
Buneman P: The recovery of trees from measurements of dissimilarity. Mathematics in the Archeological and Historical Sciences. Edinburgh: Edinburgh University Press 1971, 387395.
 14.
Estabrook GF, Johnson J, McMorris FR: A mathematical foundation for the analysis of cladistic character compatibility. Math Biosci. 1976, 29 (12): 181187. 10.1016/00255564(76)900353.
 15.
Meacham CA: Theoretical and computational considerations of the compatibility of qualitative taxonomic characters. Numerical Taxonomy, Volume G1 of Nato ASI series. Heidelberg: Springer 1983, 304314.
 16.
Fitch WM: Toward finding the tree of maximum parsimony. Proceedings of the 8th International Conference on Numerical Taxonomy. San Francisco: Freeman 1975, 189230.
 17.
Fitch WM: On the problem of discovering the most parsimonious tree. Am Nat. 1977, 111 (978): 223257. 10.1086/283157.
 18.
Habib M, To TH: On a conjecture of compatibility of multistates characters. Algorithms in Bioinformatics, Volume 6833 of Lecture Notes in Computer Science. Edited by: Przytycka T, Sagot MF. Berlin/Heidelberg: Springer 2011, 116127.
 19.
Buneman P: A characterization of rigid circuit graphs. Discrete Math. 1974, 9: 205212. 10.1016/0012365X(74)900028.
 20.
Colonius H, Schulze HH: Tree structures for proximity data. Br J Math Stat Psychol. 1981, 34 (2): 167180. 10.1111/j.20448317.1981.tb00626.x.
 21.
Dekker MCH: Reconstruction methods for derivation trees. Master’s thesis. Amsterdam: Vrije Universiteit 1986.
 22.
Dietrich M, McCartin C, Semple C: Bounding the maximum size of a minimal definitive set of quartets. Inf Process Lett. 2012, 112 (16): 651655. 10.1016/j.ipl.2012.06.001.
 23.
Grünewald S, Huber KT: Identifying and defining trees. Reconstructing Evolution: New Mathematical and Computational Advances. Edited by: Gascuel O, Steel M. Oxford University Press, 2007.
 24.
Steel M: Personal communications. 2012,
 25.
Aho AV, Sagiv Y, Szymanski TG, Ullman JD: Inferring a tree from lowest common ancestors with an application to the optimization of relational expressions. SIAM J Comput. 1981, 10 (3): 405421. 10.1137/0210030.
 26.
Bryant D, Steel M: Extension operations on sets of leaflabelled trees. Adv Appl Math. 1995, 16: 425453. 10.1006/aama.1995.1020.
Acknowledgements
We thank Sylvain Guillemot, Mike Steel, and Rob Gysel for valuable comments. This work was supported in part by the National Science Foundation under grants CCF1017189 and DEB0829674.
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
BS was responsible for the lower bounds on character compatibility, the upper and lower bounds on quartet compatibility, the characterization of threestate character compatibility, and wrote all portions of the manuscript other than the section on triplet compatibility. SV was responsible for the upper and lower bounds on triplet compatibility, contributed to the lower bounds on quartet and character compatibility, and wrote the portion of the manuscript on triplet compatibility. DFB supervised the project. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License(http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Shutters, B., Vakati, S. & FernándezBaca, D. Incompatible quartets, triplets, and characters. Algorithms Mol Biol 8, 11 (2013). https://doi.org/10.1186/17487188811
Received:
Accepted:
Published:
Keywords
 Phylogenetics
 Quartet compatibility
 Triplet compatibility
 Character compatibility
 Perfect phylogeny