DeBi: Discovering Differentially Expressed Biclusters using a Frequent Itemset Approach

Background The analysis of massive high throughput data via clustering algorithms is very important for elucidating gene functions in biological systems. However, traditional clustering methods have several drawbacks. Biclustering overcomes these limitations by grouping genes and samples simultaneously. It discovers subsets of genes that are co-expressed in certain samples. Recent studies showed that biclustering has a great potential in detecting marker genes that are associated with certain tissues or diseases. Several biclustering algorithms have been proposed. However, it is still a challenge to find biclusters that are significant based on biological validation measures. Besides that, there is a need for a biclustering algorithm that is capable of analyzing very large datasets in reasonable time. Results Here we present a fast biclustering algorithm called DeBi (Differentially Expressed BIclusters). The algorithm is based on a well known data mining approach called frequent itemset. It discovers maximum size homogeneous biclusters in which each gene is strongly associated with a subset of samples. We evaluate the performance of DeBi on a yeast dataset, on synthetic datasets and on human datasets. Conclusions We demonstrate that the DeBi algorithm provides functionally more coherent gene sets compared to standard clustering or biclustering algorithms using biological validation measures such as Gene Ontology term and Transcription Factor Binding Site enrichment. We show that DeBi is a computationally efficient and powerful tool in analyzing large datasets. The method is also applicable on multiple gene expression datasets coming from different labs or platforms.


Background
In recent years, various high throughput technologies such as cDNA microarrays, oligo-microarrays and sequence-based approaches (RNA-Seq) for transcriptome profiling have been developed. The most common approach for detecting functionally related gene sets from such high throughput data is clustering [1]. Traditional clustering methods like hierarchical clustering [2] and k-means [3], have several limitations. Firstly, they are based on the assumption that a cluster of genes behaves similarly in all samples. However, a cellular process may affect a subset of genes, only under certain conditions. Secondly, clustering assigns each gene or sample to a single cluster. However, some genes may not be active in any of the samples and some genes may participate in multiple processes.
Biclustering is a two-way clustering method for detecting local patterns in data. It finds subsets of genes that behave similarly in subsets of samples. Biclustering was initially introduced by Hartigan [4]. However, it was first applied by Cheng and Church [5] on gene expression data. Cheng and Church tried to identify submatrices of low mean residue score which indicates uniform fluctuation in expression profiles. Since the algorithm discovers one bicluster at a time, repeated application of the method on a modified matrix is needed for discovering multiple biclusters. This has the drawback that it results in highly overlapping gene sets. Ben-Dor et al. [6] detected a subset of genes whose expression levels induce the same linear ordering of the experiments. The drawback of this method is that it enforces a strict order of the samples. Bergmann et al. [7] identified biclusters which consist of the set of co-regulated genes and the conditions that induce their co-regulation. Murali and Kasif [8] found subsets of genes that are simultaneously similarly expressed across a subset of the samples. The algorithm uses prior knowledge about the sample phenotypes. Tanay et al. [9] defined biclustering as a problem of finding bicliques in a bipartite graph. Due to its high complexity, the number of rows the bicluster may have is restricted. Prelic et al. [10] defined the binary inclusion maximal biclustering (BIMAX) using a fast divide and conquer method. However, divide and conquer has the drawback of possibly missing good biclusters by early splits. Li et al. [11] developed an algorithm for discovering statistically significant biclusters from datasets containing tens of thousands of genes and thousands of conditions. Madeira and Oliveira have written a detailed review on different biclustering methods [12].
Here, we propose a novel, fast biclustering algorithm called DeBi that utilizes differential gene expression analysis. In DeBi, a bicluster has the following two main properties. Firstly, a bicluster is a maximum homogenous gene set where each gene in the bicluster should be highly or lowly expressed over all the bicluster samples. Secondly, each gene in the bicluster shows statistical difference in expression between the samples in the bicluster and the samples not in the bicluster. Differentially expressed biclusters lead to functionally more coherent gene sets compared to standard clustering or biclustering algorithms.
There are several advantages of the DeBi algorithm. Firstly, the algorithm is capable of discovering biclusters on very large datasets such as the human connectivity map data with 22283 genes and 6100 samples in reasonable time. Secondly, it is not required to define the number of biclusters a priori [5,7,10].
We evaluated the performance of DeBi on a yeast dataset [13], on synthetic datasets [10], on the connectivity map dataset which is a reference collection of gene expression profiles from human cells that have been treated with a variety of drugs [14], gene expression profiles of 2158 human tumor samples published by expO (Expression Project for Oncology), on diffuse large B-cell lymphoma (DLBCL) dataset [15] and on gene sets from the Molecular Signature Database (MSigDB) C2 category. We show that DeBi compares well with existing biclustering methods such as BIMAX, SAMBA, Cheng and Church's algorithm (CC), Order Preserving Submatrix Algorithm (OPSM), Iterative Signature Algorithm (ISA) and Qualitative Biclustering (QUBIC) [5][6][7]9,10].

Results
We have evaluated our algorithm on six datasets (a) Prelic's benchmark synthetic datasets with implanted biclusters [10] (b) 300 different experimental perturbations of S. cerevisiae [13] (c) diffuse large B-cell lymphoma (DLBCL) dataset [15] (d) a reference collection of gene-expression profiles from human cells that have been treated with a variety of drugs [14] (e) gene expression profiles of 2158 human tumor samples published by expO (Expression Project for Oncology) (f) gene sets from the Molecular Signature Database (MSigDB) C2 category. The synthetic data is studied to show the performance of our algorithm in recovering implanted biclusters. Additionally, the effect of overlap between biclusters and noise on the performance of the algorithm can be studied using the synthetic data. The yeast and human gene expression datasets are studied to evaluate the biological relevance of the biclusters from several aspects. We used a fold-change of 2 for binarizing the datasets. The set of biclusters generated by all the algorithms are filtered such that the remaining ones have a maximum overlap of 0.5. (unless specified otherwise) First, for each bicluster we calculated the statistically significantly enriched Gene Ontology (GO) terms using the hypergeometric test. We determined the proportion of GO term enriched biclusters at different levels of significance. Second, Transcription Factor Binding Sites (TFBS) enrichment is calculated by a hypergeometric test using transcription factor binding site data coming from various sources [16][17][18] at different levels of significance. The GO term and TFBS enrichment analyses are done using Genomica http://genie.weizmann.ac.il.

Prelic's Synthetic Data
We applied our algorithm to a synthetic gene expression dataset. In the artificial datasets biclusters have been created on the basis of two scenarios (data available at http://www.tik.ee.ethz.ch/sop/bimax. In the first scenario, non-overlapping biclusters with increasing noise levels are generated. In the second scenario, biclusters with increasing overlap but without noise are produced. In both scenarios, biclusters with constant expression values and biclusters following an additive model where the expression values varying over the conditions are investigated.
In order to assess the performance of different biclustering algorithms, we used two measures from Prelic et al. [10] and Hochreiter et al. [21], respectively. The measure introduced by Prelic et al. calculates a similarity based on the Jaccard index between the computed biclusters and the implanted biclusters. Bicluster recovery score measures the accuracy of the predicted biclusters however it does not consider the number of biclusters in both sets. Hochreiter et al. introduced a consensus score by computing similarities between all pairs of biclusters and then assigning the biclusters of one set to biclusters of the other set. It penalizes different number of biclusters by dividing the sum of similarities by the numbers of biclusters in largest set. A more detailed description of the measures can be found in Additional File 1.
In Figures 1 and 2 the performance of BIMAX, ISA, SAMBA, DeBi, OPSM and QUBIC algorithms on the synthetic data is summarized based on Prelic et al. recovery score and Hochreiter et al. consensus score. The set of biclusters generated by these algorithms are filtered such that the remaining ones have a maximum overlap of 0.25. In the Prelic et al. paper, after the filtering process the largest 10 biclusters are chosen. Since the bicluster number is not known a priori, we have considered all the filtered biclusters. We did not evaluate xMotif and CC algorithms since they have been shown to perform badly in all the scenarios, mostly below 50% of recovery accuracy [10]. The CC and xMotif algorithms produce large biclusters containing genes that are not expressed. ISA and QUBIC give high  Figure S1 in Additional File 1. In the absence of overlap with increasing noise levels, DeBi is able to identify 99% of implanted biclusters both in additive and constant model. High degree of overlap decreases the performance of DeBi because it considers the overlapping part of the biclusters as a seperate bicluster. The DeBi biclustering results can be found in Additional file 2.

Yeast Compendium
We further applied our algorithm to the compendium of gene expression profiles derived from 300 different experimental perturbations of S. cerevisiae [13]. We discovered 192 biclusters in the yeast dataset containing 2025 genes and 192 conditions. As a binarization level we used the fold change of 1.58 as recommended in the original paper [13]. Figure 3 (a) illustrates the proportion of GO term and TFBS enriched biclusters for the six selected biclustering methods (ISA, OPSM, BIMAX, QUBIC, SAMBA and DeBi) at different levels of significance. DeBi performs the second best based on biological validation measures. BIMAX discovers a higher proportion of GO term and TFBS enriched biclusters. All the biclusters, the enrichment analysis can be found in Additional file 3.
In the analyzed yeast data, conditions are knocked-out genes. Since biclustering discovers subsets of genes and subsets of conditions we can also examine the biological significance of the clustered conditions. Similar to the previous analysis, we measured GO term enrichment of conditions in each discovered biclusters. DeBi is the second best in discovering high percentage of GO term enriched biclusters.
In the discovered biclusters, the enriched gene functions are related to the enriched sample functions. Bicluster 83, genes are enriched in the 'conjugation' GO term and conditions are enriched in 'regulation of biological quality'. Moreover, there is an enrichment of the TFBS of STE12, which is known to be involved in cell cycle. Bicluster 50, consists of genes and samples that are enriched in 'ribosome biogenesis and assembly' GO term. Bicluster 22, consists of genes and samples that are enriched in 'lipid metabolic process' GO term, and additionally genes are enriched with TFBS of HAP1. Bicluster 9, consists of down regulated genes and samples that are enriched in 'cell division' GO term, and additionally genes are enriched with TFBS of STE12.

DLBCL Data
We also evaluated our DeBi algorithm on 'diffuse large B-cell lymphoma' (DLBCL) dataset. DLBCL dataset consists of 661 genes and 180 samples. We applied ISA, OPSM, QUBIC, SAMBA and DeBi algorithms. Figure 3 (b) illustrates the proportion of GO term and TFBS enriched biclusters for the five biclustering methods at different levels of significance. DeBi discovers the highest proportion of GO term and TFBS enriched biclusters. The up regulated bicluster 16 and down regulated bicluster 4 contains the sample classes identified by [22]. Bicluster 16 is enriched with 'ribosome' and 'cell cycle' GO Term and Bicluster 4 is enriched with 'cell cycle' and 'death' GO Terms. The protein interaction networks of this two selected biclusters can be found in Figure S2 and S3 Additional File 1. Protein interaction networks are generated using STRING [23]. All the biclusters and the enrichment analysis can be found in Additional file 4.

Human CMap Data
We also evaluated our DeBi algorithm on the Connectivity Map v0.2 (CMap) [14]. CMap is a reference collection of gene expression profiles from human cells that have been treated with a variety of drugs comprised of 6100 samples and 22283 genes. Figure 3 (c) summarizes the results of DeBi and QUBIC. The proportion of GO term and TFBS enriched biclusters are much more higher in DeBi compared to QUBIC.
The biclusters discovered by DeBi can be used to find drugs with a common mechanism of action and identify new therapeutics. Moreover, we can observe the effect of drugs on different cell lines. Figure 4 shows parallel coordinate plots of some of the identified biclusters. In parallel coordinate plots, the profile of the conditions that are included in a bicluster are shown as black, the other conditions as gray. This aids to visualize the expression difference between the conditions in a bicluster compared to the rest of the conditions. The bicluster 6, contains up regulated 'heat shock protein binding' genes and 'heat shock protein inhibitors' such as geldanamycin, alvespimycin, tanespimycin, monorden. Heat shock proteins (Hsps) are overexpressed in a wide range of human cancers and are involved in tumor cell  Additionally, protein interaction networks of the selected biclusters are strikingly connected and they can be found in Figure S4, S5, S6 and S7 in Additional File 1. All the biclusters and the enrichment analysis can be found in Additional file 5.

Human ExpO Data
We   Figure 3 (d) shows that the proportion of GO term and TFBS enriched biclusters are much more higher in DeBi compared to QUBIC. It illustrates that DeBi performs better than QUBIC in ExpO data. 70% of the DeBi biclusters are enriched with GO Terms with a p-value smaller than 0.05. Moreover biclusters contain tumor samples mostly from similar tissue types. Figure  S8 in Additional file 1 shows GO Term enrichment of some of the biclusters. Bicluster 13 contains thyroid tumor samples and genes enriched with 'protein-hormone receptor activity'. Bicluster 3 contains prostate tumor samples and genes enriched with 'tissue kallikrein activity'. Bicluster 22 contains mostly pancreas and colon samples and genes enriched with 'pancreatic elastase activity' GO Term. All the biclusters and the enrichment analysis can be found in Additional file 6.

MSigDB Data
Finally, we applied our algorithm on the manually curated gene sets from the Molecular Signature Database    , illustrates all the biclusters using BiVoc algorithm [25]. BiVoc algorithm rearranges rows and conditions in order to represent the biclusters with the minimum space. The output matrix of BiVoc, may have repeated rows and/or columns from the original matrix. In Figure 5, the function of each bicluster is specified based on GO Term enrichment. Bicluster 3, contains the down-regulated gene set from Alzheimer patients and gene set from proteasome pathway. It is known that there is a significant decrease in proteasome activity in Alzheimer patients [26]. Bicluster 3 also contains the In previous studies, high activity of ubiquitin-proteasome pathway in pancreatic cancer cell line was detected [27]. Bicluster 8 contains up-regulated gene set from liver cancer patients and gene set from G-protein activation pathway. Dysfunction of G Protein-Coupled Receptor signaling pathways are involved in certain forms of cancer. All the biclusters and the enrichment analysis can be found in Additional file 7.

Running Time
DeBi algorithm is capable of analyzing yeast data(size 6100 × 300) in 6 minutes, ExpO data (size 40223 × 2158) in 12 minutes, MSigDB data (size 18205 × 3272) in 11 minutes, DLBCL data (size 610 × 180) in 11 seconds, CMap data (size 22283 × 6100) in 3 hours 45 minutes. The QUBIC algorithm analyzes CMap data in 2 hours 55 minutes and ExpO data in 3 hours 54 minutes. The running time analysis was done on a 2.13 GHz Intel 2 Dual Core computer with 2GB memory.

Methods
Given an expression matrix E with genes G ={g 1 , g 2 , g 3 ,..., g n } and samples S ={s 1 , s 2 , s 3 ,..., s m } a bicluster is defined as b = (G', S') where G' ⊂ G is a subset of genes and S' ⊂ S is a subset of samples. DeBi identifies functionally coherent biclusters B ={b 1 , b 2 , b 3 ,..., b l } in three steps. Below we describe each step in detail. An overview of the DeBi algorithm is shown in Figure 6. The DeBi algorithm is based on a well known data mining approach called Maximal Frequent Item Set [28]. We will refer to this as Maximal Frequent Gene Set, as given by our problem definition. The pseudocode of the algorithm is in Additional file 1.

Preliminaries
The input gene expression data is binarized according to either up or down regulation. Let E u and E d denote the up and down regulation binary matrices, respectively. Then the entries e u ij of E u are defined as follows: and the entries of e d ij of E d are defined analogously with a c-fold down-regulation cut-off. The fold change cut-off c will typically be set to 2.

Finding seed biclusters by Maximal Frequent Gene Set Algorithm
The DeBi algorithm, identifies the seed gene sets by iteratively applying the maximal frequent gene set algorithm. We first define the term support, which we will later use in the algorithm. The support of the gene g i , i = 1 ,..., n, is defined as follows: In other words, the support is the proportion of samples for which the gene-vector e i . is 1. This is further extended to sets of genes. Let G v = {g 1 , . . . , g k } be the v th gene-set. For a set of gene-vectors we define their phenotype vector C v as their element-wise logical AND: The support of the gene set is then defined as the fraction of samples for which the phenotype vector is 1.
A gene set G v is (c 1 , c 2 )frequent iff its support supp (G v ) is larger than c 1 and the cardinality |G v | above c 2 . Serin and Vingron Algorithms for Molecular Biology 2011, 6:18 http://www.almob.org/content/6/1/18 Figure 6 Illustration of DeBi algorithm. The algorithm is ran on two different binarized datasets. One is the binarized data based on up regulation and the other is the binarized data based on down regulation. In Step 1, seed biclusters identified within each support value going from high to low. For the binarized data based on up regulation, in the 1st iteration, red gene set with support value 10/20 is detected and excluded from the search space. Similarly, in the second and third iterations yellow and blue clusters with support values, respectively 6/20 and 4/20, are found. In Step 2, seed gene sets are improved based on genes' association strength. Gene 15 is added to the red bicluster because the p-value returned by the Fisher exact test is smaller than a and gene 13 is deleted because the p-value returned by the Fisher exact test is higher than a. None of the discovered biclusters have an overlap of the gene × sample area of more than 50%.
Serin and Vingron Algorithms for Molecular Biology 2011, 6:18 http://www.almob.org/content/6/1/18 When c 1 and c 2 are not in focus, we will simply speak of a frequent gene set. A gene set is maximally frequent iff it is frequent and no superset of it is frequent.
The simplest method for detecting maximally frequent gene sets is a brute force approach in which each possible subset of G ={g 1 , g 2 , g 3 ,..., g n } is a candidate frequent set. To find the frequent sets we count the support of each candidate set. The MAFIA algorithm is an efficient implementation for finding maximally frequent sets with support above a given threshold [28]. The search strategy of MAFIA uses a depth-first traversal of the gene set lattice with effective pruning techniques. It avoids exhaustive enumeration of all candidate gene sets by a monotonicity principle. The monotonicity principle states that every subset of a frequent itemset is frequent. It prunes the candidates which have an infrequent subpattern using this property.
In the first step of the DeBi algorithm, MAFIA is iteratively applied to the binary matrix successively reducing the support threshold. Initially, MAFIA is applied to the full binary matrix E u (E d ) with support value (c 1 ) 0 equal to support value of the gene with the highest support. In iteration k, MAFIA is applied with support value threshold of (c 1 identified maximally frequent sets are added to the set of seed gene sets B and the genes in B are deleted from the binary matrix E u (E d ). In each iteration MAFIA is applied to the modified matrix E u (E d ). The process is repeated until a user defined minumum support parameter is reached.
The entries of C v indicate the indices of the bicluster samples. If C vj = 1 ⇒ s j ∈ S v , j = 1,...,m , i.e. that the sample s j belongs to the bicluster b v . The gene g i , i = 1,..., n, is an element of gene set G v if e i . is associated with C v . We evaluate the association strength between the phenotype vector of a bicluster and another gene using Fisher's exact test on a 2 × 2 contingency table.
The cells of the contingency table count how often the four possibilities of the phenotype vector containing a 1 or a 0 and the gene-vector containing a 1 or a 0 occur. The Fisher's exact test then tests for independence in the contingency table and thus among the two vectors. A gene g i , i = 1,..., n is added, to the gene set G v if the pvalue p g i returned by the Fisher exact test is lower than the parameter a. It gets deleted from b v if the probability is higher than a and added to b v if the probability is smaller than a. For this procedure the association probability p g i with the bicluster needs to be calculated for each gene. However, we reduce the computational effort using the monotonicity property of the hypergeometric distribution. We precompute cut-off values on the contingency table entries that yield a p-value just higher than a. Let s 1, IN and s 1, OUT denote the number of 1's a gene-vector has in the bicluster samples and the number of 1's a gene-vector has outside the bicluster samples, respectively. We find the minimal s 1, IN and maximal s 1, OUT at this border. Then, we apply Fisher's exact test only to those genes which have s 1, IN >mins 1, IN and s 1, OUT <maxs 1, OUT .
In the last step we turn to the sometimes very complicated overlap structure among biclusters. The goal is to filter the set of biclusters such that the remaining ones are large and overlap only little. The size of a bicluster is defined as the number of genes times the number of samples in the bicluster, |G v | × |S v |. Two biclusters overlap when they share common samples and genes. The size of the overlap is the product of the number of common samples and common genes. To filter out biclusters that are largely contained in a bigger bicluster, we start with the largest bicluster and compare it to the other biclusters. Those biclusters for which the overlap to the largest one exceeds L% (typically 50%) of the size of the smaller one are deleted. This is then repeated starting with the remaining second-largest bicluster and so on.

Choosing the optimum alpha parameter
To formulate an optimality criterion for a one requires an inherent measure of the quality of a set of biclusters. To this end, for a bicluster v, we define its score I v as the negative sum of the log p-values of the included genes, where the individual p g is the p-value from the Fisher exact test: However, this bicluster score I v depends on the size (number of genes × number of conditions) of the bicluster and in order to make it comparable between biclusters one needs to correct for the size. We compute the expected bicluster score through a randomization procedure. A large number, say 500, random phenotype vectors having the same number of 1s as the bicluster has conditions is generated. For these random phenotype vectors a Fisher exact test p-value with respect to each gene in the bicluster is computed. One obtains a random I v score by adding log p-values over the genes of the bicluster. The mean of these random bicluster scores is the desired estimator. Finally, a normalized NI v score is definded by dividing I v by this estimated mean and the total biclustering score CS is defined as the sum of NI v normalized scores of all discovered biclusters CS = vε I (NI v ). This score serves to distinguish between different choices of a. The program is run under a = {10 -2 , 10 -3 ,..., 10 -100 } and we choose the a that maximizes CS.

Discussion
We have proposed a novel fast biclustering algorithm especially for analyzing large datasets. Our algorithm aims to find biclusters where each gene in a bicluster should be highly or lowly expressed over all the bicluster samples compared to the rest of the samples. Unlike other algorithms, it is not required to define the number of biclusters a priori. We have compared our method with other biclustering algorithms using synthetic data and biological data. We have shown that the DeBi algorithm provides biologically significant biclusters using GO term and TFBS enrichment. We have also showed the computational efficiency of our algorithm. It is shown that it is a useful and powerful tool in analyzing large datasets.
In spite of efforts by many authors, comparing the performance of biclustering algorithms is still a challenge [29]. Smaller biclusters have a higher chance to yield a coherent GO annotation, while larger biclusters would, of course, be more interesting to observe. Our a threshold influences this behavior. The optimized α threshold is smaller for larger number of samples which limits the number of genes that get accepted into a bicluster.
The binarization of the input data in order to obtain a boolean matrix is another key decision in our approach. In this we go along with many other authors and we think that it helps in applying biclustering to gene expression data coming from different labs or platforms. The hope is that our method will further contribute to establishing biclustering as a general purpose tool for data analysis in functional genomics.