A linear programming approach for estimating the structure of a sparse linear genetic network from transcript profiling data
 Sahely Bhadra^{1},
 Chiranjib Bhattacharyya^{1, 2}Email author,
 Nagasuma R Chandra^{2}Email author and
 I Saira Mian^{3}
https://doi.org/10.1186/1748718845
© Bhadra et al; licensee BioMed Central Ltd. 2009
Received: 30 May 2008
Accepted: 24 February 2009
Published: 24 February 2009
Abstract
Background
A genetic network can be represented as a directed graph in which a node corresponds to a gene and a directed edge specifies the direction of influence of one gene on another. The reconstruction of such networks from transcript profiling data remains an important yet challenging endeavor. A transcript profile specifies the abundances of many genes in a biological sample of interest. Prevailing strategies for learning the structure of a genetic network from highdimensional transcript profiling data assume sparsity and linearity. Many methods consider relatively small directed graphs, inferring graphs with up to a few hundred nodes. This work examines large undirected graphs representations of genetic networks, graphs with many thousands of nodes where an undirected edge between two nodes does not indicate the direction of influence, and the problem of estimating the structure of such a sparse linear genetic network (SLGN) from transcript profiling data.
Results
The structure learning task is cast as a sparse linear regression problem which is then posed as a LASSO (l_{1}constrained fitting) problem and solved finally by formulating a Linear Program (LP). A bound on the Generalization Error of this approach is given in terms of the LeaveOneOut Error. The accuracy and utility of LPSLGNs is assessed quantitatively and qualitatively using simulated and real data. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) initiative provides gold standard data sets and evaluation metrics that enable and facilitate the comparison of algorithms for deducing the structure of networks. The structures of LPSLGNs estimated from the IN SILICO 1, IN SILICO 2 and IN SILICO 3 simulated DREAM2 data sets are comparable to those proposed by the first and/or second ranked teams in the DREAM2 competition. The structures of LPSLGNs estimated from two published Saccharomyces cerevisae cell cycle transcript profiling data sets capture known regulatory associations. In each S. cerevisiae LPSLGN, the number of nodes with a particular degree follows an approximate power law suggesting that its degree distributions is similar to that observed in realworld networks. Inspection of these LPSLGNs suggests biological hypotheses amenable to experimental verification.
Conclusion
A statistically robust and computationally efficient LPbased method for estimating the topology of a large sparse undirected graph from highdimensional data yields representations of genetic networks that are biologically plausible and useful abstractions of the structures of real genetic networks. Analysis of the statistical and topological properties of learned LPSLGNs may have practical value; for example, genes with high random walk betweenness, a measure of the centrality of a node in a graph, are good candidates for intervention studies and hence integrated computational – experimental investigations designed to infer more realistic and sophisticated probabilistic directed graphical model representations of genetic networks. The LPbased solutions of the sparse linear regression problem described here may provide a method for learning the structure of transcription factor networks from transcript profiling and transcription factor binding motif data.
Keywords
Background
Understanding the dynamic organization and function of networks involving molecules such as transcripts and proteins is important for many areas of biology. The ready availability of highdimensional data sets generated using highthroughput molecular profiling technologies has stimulated research into mathematical, statistical, and probabilistic models of networks. For example, GEO [1] and ArrayExpress [2] are public repositories of wellannotated and curated transcript profiling data from diverse species and varied phenomena obtained using different platforms and technologies.
A genetic network can be represented as a graph consisting of a set of nodes and a set of edges. A node corresponds to a gene (transcript) and an edge between two nodes denotes an interaction between the connected genes that may be linear or nonlinear. In a directed graph, the oriented edge A → B signifies that gene A influences gene B. In an undirected graph, the unoriented edge A  B encodes a symmetric relationship and signifies that genes A and B may be coexpressed, coregulated, interact or share some other common property. Empirical observations indicate that most genes are regulated by a small number of other genes, usually fewer than ten [3–5]. Hence, a genetic network can be viewed as a sparse graph, i.e., a graph in which a node is connected to a handful of other nodes. If directed (acyclic) graphs or undirected graphs are imbued with probabilities, the result is probabilistic directed graphical models and probabilistic undirected graphical models respectively [6].
Extant approaches for deducing the structure of genetic networks from transcript profiling data [7–9] include Boolean networks [10–14], linear models [15–18], neural networks [19], differential equations [20], pairwise mutual information [10, 21–23], Gaussian graphical models [24, 25], heuristic approachs [26, 27], and coexpression clustering [16, 28]. Theoretical studies of sample complexity indicate that although sparse directed acyclic graphs or Boolean networks could be learned, inference would be problematic because in current data sets, the number of variables (genes) far exceedes the number of observations (transcript profiles) [12, 14, 25]. Although probabilistic graphical models provide a powerful framework for representing, modeling, exploring, and making inferences about genetic networks, there remain many challenges in learning tabula rasa the topology and probability parameters of large, directed (acyclic) probabilistic graphical models from uncertain, highdimensional transcript profiling data [7, 25, 29–33]. Dynamic programing approaches [26, 27] use Singular Value Decomposition (SVD) to preprocess the data and heuristics to determine stopping criteria. These methods have high computational complexity and yield approximate solutions.
This work focuses on a plausible, albeit incomplete representation of a genetic network – a sparse undirected graph – and the task of estimating the structure of such a network from highdimensional transcript profiling data. Since the degree of every node in a sparse graph is small, the model embodies the biological notion that a gene is regulated by only a few other genes. An undirected edge indicates that although the expression levels of two connected genes are related, the direction of influence is not specified. The final simplification is that of restricting the type of interaction that can occur between two genes to a single class, namely a linear relationship. This particular representation of a genetic network is termed a sparse linear genetic network (SLGN).
Here, the task of learning the structure of a SLGN is equated with that of solving a collection of sparse linear regression problems, one for each gene in a network (node in the graph). Each linear regression problem is posed as a LASSO (l_{1}constrained fitting) problem [34] that is solved by formulating a Linear Program (LP). A virtue of this LPbased approach is that the use of the Huber loss function reduces the impact of variation in the training data on the weight vector that is estimated by regression analysis. This feature is of practical importance because technical noise arising from the transcript profiling platform used coupled with the stochastic nature of gene expression [35] leads to variation in measured abundance values. Thus, the ability to estimate parameters in a robust manner should increase confidence in the structure of an LPSLGN estimated from noisy transcript profiles. An additional benefit of the approach is that the LP formulations can be solved quickly and efficiently using widely available software and tools capable of solving LPs involving tens of thousands of variables and constraints on a desktop computer.
Two different LP formulations are proposed: one based on a positive class of linear functions and the other on a general class of linear functions. The accuracy of this LPbased approach for deducing the structure of networks is assessed statistically using gold standard data and evaluation metrics from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) initiative [36]. The LPbased approach compares favourably with algorithms proposed by the top two ranked teams in the DREAM2 competition. The practical utility of LPSLGNs is examined by estimating and analyzing network models from two published Saccharomyces cerevisiae transcript profiling data sets [37] (ALPHA; CDC15). The node degree distributions of the learned S. cerevisiae LPSLGNs, undirected graphs with many hundreds of nodes and thousands of edges, follow approximate power laws, a feature observed in real biological networks. Inspection of these LPSLGNs from a biological perspective suggests they capture known regulatory associations and thus provide plausible and useful approximations of real genetic networks.
Methods
Genetic network: sparse linear undirected graph representation
A genetic network can be viewed as an undirected graph, $\mathcal{G}$ = {V, W}, where V is a set of N nodes (one for each gene in the network), and W is an N × N connectivity matrix encoding the set of edges. The (i, j)^{ th }element of the matrix W specifies whether nodes i and j do (W_{ ij }≠ 0) or do not (W_{ ij }= 0) influence each other. The degree of node n, k_{ n }, indicates the number of other nodes connected to n and is equivalent to the number of nonzero elements in row n of W. In real genetic networks, a gene is regulated often by a small number of other genes [3, 4] so a reasonable representation of a network is a sparse graph. A sparse graph is a graph $\mathcal{G}$ parametrized by a sparse matrix W, a matrix with few nonzero elements W_{ ij }, and where most nodes have a small degree, k_{ n }< 10.
Linear interaction model: static and dynamic settings
If the relationship between two genes is restricted to the class of linear models, the abundance value of a gene is treated as a weighted sum of the abundance values of other genes. A highdimensional transcript profile is a vector of abundance values for N genes. An N × T matrix E is the concatenation of T profiles, [e(1),..., e(T)], where e(t) = [e_{1}(t),..., e_{ N }(t)]^{⊤} and e_{ n }(t) is the abundance of gene n in profile t. In most extant profiling studies, the number of transcripts monitored exceeds the number of available profiles (N ≫ T).
The parameter w_{ n }= [w_{n 1},..., w_{ nN }]^{⊤} is a weight vector for gene n and the j^{ th }element indicates whether genes n and j do (w_{ nj }≠ 0) or do not (w_{ nj }= 0) influence each other. The constraint w_{ nn }= 0 prevents gene n from influencing itself at the same instant so its abundance is a function of the abundances of the remaining N  1 genes in the same profile.
In the dynamic setting, the T transcript profiles in E are assumed to form a time series. In the linear interaction model, the abundance value of a gene at time t is treated as a weighted sum of the abundance values of all genes in the profile from the previous time point, t  1, i.e., ${e}_{n}(t)={w}_{n}^{\text{T}}e(t1)$. There is no constraint w_{ nn }= 0 because a gene can influence its own abundance at the next time point.
As described in detail below, the SLGN structure learning problem involves solving N independent sparse linear regression problems, one for each node in the graph (gene in the network), such that every weight vector w_{ n }is sparse. The sparse linear regression problem is cast as an LP and uses a loss function which ensures that the weight vector is resilient to small changes in the training data. Two LPs are formulated and each formulation contains one userdefined parameter, A, the upper bound of the l_{1} norm of the weight vector. One LP is based on a general class of linear functions. The other LP formulation is based on a positive class of linear functions and yields an LP with fewer variables than the first.
Simulated and real data
DREAM2 InSilicoNetwork Challenges data
A component of Challenge 4 of the DREAM2 competition [38] is predicting the connectivity of three in silico networks generated using simulations of biological interactions. Each DREAM2 data set includes time courses (trajectories) of the network recovering from several external perturbations. The IN SILICO 1 data were produced from a gene network with 50 genes where the rate of synthesis of the mRNA of each gene is affected by the mRNA levels of other genes; there are 23 different perturbations and 26 time points for each perturbation. The IN SILICO 2 data are similar to IN SILICO 1 but the topology of the 50gene network is qualitatively different. The IN SILICO 3 data were produced from a full in silico biochemical network that had 16 metabolites, 23 proteins and 20 genes (mRNA concentrations); there are 22 different perturbations and 26 time points for each perturbation. Since the LPbased method yields network models in the form of undirected graphs, the data were used to make predictions in the DREAM2 competition category UNDIRECTEDUNSIGNED. Thus, the simulated data sets used to estimate LPSLGNs are an N = 50 × T = 26 matrix (IN SILICO 1), an N = 50 × T = 26 matrix (IN SILICO 2), and an N = 59 × T = 26 matrix (IN SILICO 3).
S. cerevisiae transcript profiling data
A published study of S. cerevisiae monitored 2,467 genes at various time points and under different conditions [37]. In the investigations designated ALPHA and CDC15, measurements were made over T = 15 and T = 18 time points respectively. Here, a gene was retained only if an abundance measurement was present in all 33 profiles. Only 605 genes met this criterion of no missing values and these data were not processed any further. Thus, the real transcript profiling data sets used to estimate LPSLGNs are an N = 605 × T = 15 matrix (ALPHA) and an N = 605 × T = 18 matrix (CDC15).
Training data for regression analysis
A training set for regression analysis, ${\left\{{\mathcal{D}}_{n}\right\}}_{n=1}^{N}$, is created by generating training points for each gene from the data matrix E. For gene n, the training points are ${\mathcal{D}}_{n}={\{({x}_{ni},{y}_{ni})\}}_{i=1}^{I}$. The i^{ th }training point consists of an "input" vector, x_{ ni }= [x_{1i},..., x_{ Ni }] (abundances values for N genes), and an "output" scalar y_{ ni }= x_{ ni }(abundance value for gene n).
In the static setting, I = T training points are created because both the input and output are generated from the same profile; the linear interaction model (Equation 1) includes the constraint w_{ nn }= 0. If e_{ n }(t) is the abundance of gene n in profile t, the i^{ th }training point is x_{ ni }= e(t) = [e_{1}(t),..., e_{ N }(t)], y_{ ni }= e_{ n }(t), and t = 1,..., T.
In the dynamic setting, I = T  1 training points are created because the output is generated from the profile for a given time point whereas the input is generated from the profile for the previous time point; there is no constraint w_{ nn }= 0 in the linear interaction model. The i^{ th }training point is x_{ ni }= e(t  1) = [e_{1}(t  1),..., e_{ N }(t  1)], y_{ ni }= e_{ n }(t), and t = 2,..., T.
The results reported below are based on training data generated under a static setting so the constraint w_{ nn }= 0 is imposed.
Notation
Let ${\mathcal{R}}^{N}$ denote the Ndimensional Euclidean vector space and card(A) the cardinality of a set A. For a vector x = [x_{1},..., x_{ N }]^{⊤} in this space, the l_{2} (Euclidean) norm is the square root of the sum of the squares of its elements, ${\Vert x\Vert}_{2}=\sqrt{{\displaystyle {\sum}_{n=1}^{N}{x}_{n}^{2}}}$; the l_{1} norm is the sum of the absolute values of its elements, ${\Vert x\Vert}_{1}={\displaystyle {\sum}_{n=1}^{N}\left{x}_{n}\right}$; and the l_{0} norm is the total number of nonzero elements, x_{0} = card({nx_{ n }≠ 0; 1 ≤ n ≤ N}). The term x ≥ 0 signifies that every element of the vector is zero or positive, x_{ n }≥ 0, ∀n ∈ {1,..., N}. The one and zerovectors are 1 = [1_{1},..., 1_{ N }]^{⊤} and 0 = [0_{1},..., 0_{ N }]^{⊤} respectively.
Sparse linear regression: an LPbased formulation
the sparse linear regression problem is the task of inferring a sparse weight vector, w_{ n }, under the assumption that genegene interactions obey a linear model, i.e., the abundance of a gene n, y_{ ni }= x_{ n }, is a weighted sum of the abundances of other genes, ${y}_{ni}={w}_{n}^{\text{T}}{x}_{ni}$.
Sparse weight vector estimation
l_{0} norm minimization
The problem of learning the structure of an SLGN involves estimating a weight vector such that w best approximates y and most of elements of w are zero. Thus, one strategy for obtaining sparsity is to stipulate that w should have at most k nonzero elements, w_{0} ≤ k. The value of k is equivalent to the degree of the node so a biologically plausible constraint for a genetic network is w_{0} ≤ 10. Given a value of k, the number of possible choices of predictors that must be examined is ^{ N }C_{ k }. Since there are many genes (N is large) and each choice of predictor variables requires solving an optimization problem, learning a sparse weight vector using an l_{0} normbased approach is prohibitive, even for small k. Furthermore, the problem is NPhard [39] and cannot even be approximated in time ${2}^{{\mathrm{log}}^{1\epsilon}N}$ where ϵ is small positive quantity.
LASSO
This formulation attempts to choose w such that it minimizes deviations between the predicted and the actual values of y. In particular, w is chosen to minimize the loss function $L(w)={\displaystyle {\sum}_{i=1}^{I}{w}^{\text{T}}{x}_{i}{y}_{i}}$. Here, "Empirical Error" is used as the loss function. The Empirical Error of a graph $\mathcal{G}$ is $\frac{1}{N}{\displaystyle {\sum}_{n=1}^{N}Empirica{l}_{error}}({\mathcal{D}}_{n})$, where $Empirica{l}_{error}({\mathcal{D}}_{n})=\frac{1}{I}{\displaystyle {\sum}_{i=1}^{I}{y}_{ni}f({x}_{ni};{w}_{n})}$. The userdefined parameter A controls the upper bound of the l_{1} norm of the weight vector and hence the tradeoff between sparsity and accuracy. If A = 0, the result is a poor approximation, as the most sparse solution is a zero weight vector, w = 0. When A = ∞, deviations are not allowed and a nonsparse w is found if the problem is feasible.
LP formulation: general class of linear functions
Consider the robust regression function f(.; w). For the general class of linear functions, f(x; w) = w^{⊤}x, an element of the parameter vector can be zero, w_{ j }= 0, or nonzero, w_{ j }≠ 0. When w_{ j }> 0, the predictor variable j makes a positive contribution to the linear interaction model, whereas if w_{ j }< 0, the contribution is negative. Since the representation of a genetic network considered here is an undirected graph and thus the connectivity matrix is symmetric, the interactions (edges) in a SLGN are not categorized as activation or inhibition.
by substituting w = u  v, w_{1} = (u + v)^{⊤}1, v_{ i } = ξ_{ i }+ ${\xi}_{i}^{*}$ and v_{ i }= ξ_{ i } ${\xi}_{i}^{*}$. The userdefined parameter A controls the upper bound of the l_{1} norm of the weight vector and thus the tradeoff between sparsity and accuracy. Problem (4) is an LP in (2N + 2I) variables, I equality constraints, 1 inequality constraints and (2N + 2I) nonnegativity constraints.
LP formulation: positive class of linear functions
Problem (5) is an LP with (N + 2I) variables, I equality constraints, 1 inequality constraints, and (2N + 2I) nonnegativity constraints.
In most transcript profiling studies, the number of genes monitored is considerably greater than the number of profiles produced, N ≫ I. Thus, an LP based on a restrictive positive linear class of functions and involving (N + 2I) variables (Problem (5)) offers substantial computational advantages over a formulation based on a general linear class of functions and involving (2N + 2I) variables (Problem (4)). LPs involving thousands of variables can be solved efficiently using extant software and tools.
To estimate a graph $\mathcal{G}$, the training points for the n^{ th }gene, ${\mathcal{D}}_{n}$, are used to solve a sparse linear regression problem posed as a LASSO and formulated as an LP. The outcome of such regression analysis is a sparse weight vector w_{ n }whose small number of nonzero elements specify which genes influence gene n. Aggregating the N sparse weight vectors produced by solving N independent sparse linear regression problems [w_{1},..., w_{ N }], yields the matrix W that parameterizes the graph.
Statistical assessment of LPSLGNs: Error, Sparsity and LeaveOneOut (LOO) Error
where w_{ n }_{0} is the l_{0} norm of the weight vector for node n.
Unfortunately, the small number of available training points (I) means that the empirical error will be optimistic and biased. Consequently, the LeaveOneOut (LOO) Error is used to analyze the stability and generalization performance of the method proposed here.
Given a training set ${\mathcal{D}}_{n}$ = [(x_{n 1}, y_{n 1}),..., (x_{ nI }, y_{ nI })], two modified training sets are built as follows

Remove the i th element: ${\mathcal{D}}_{n}^{\backslash i}={\mathcal{D}}_{n}\backslash \{({x}_{ni},{y}_{ni})\}$

Change the i th element: ${\mathcal{D}}_{n}^{i}={\mathcal{D}}_{n}\backslash \{({x}_{ni},{y}_{ni})\}\cup ({x}^{\prime},{y}^{\prime})$, where (x', y') is any point other than one in the training set ${\mathcal{D}}_{n}$
The parameter ${w}_{n}^{\backslash i}$ of the function ${f}^{\backslash i}({x}_{ni};{w}_{n}^{\backslash i})$ is learned using the modified training set ${\mathcal{D}}_{n}^{\backslash i}$.
A bound for the Generalization Error of a graph
A key issue in the design of any machine learning system is an algorithm that has low generalization error.
Here, the LeaveOneOut (LOO) error is utilized to estimate the accuracy of the LPbased algorithm employed to learn the structure of a SLGN. In this section, a bound on the generalization error based on the LOO Error is derived. Furthermore, a low "LOO Error" of the method proposed here is shown to signify good generalization.
The approch is based on the following Theorem (for details, see [43]),
Elsewhere [44], the above was given as Theorem 2.
where t is the l_{1} norm of the weight vector w_{1}. LOO Error and Error are calculated using Equation 7 and Equation 8 respectively.
PROOF. "Random draw" means that if the algorithm is run for different graphs, one graph from the set of learned graphs is selected at random. The proposed bound of generalization error will be true for this graph with high probability. This term is unrelated to term "Random graph" used in Graph Theory.
Given this bound on the generalization error, a low LOO Error in the method proposed here signifies good generalization. □
Implementation and numerical issues
Prototype software implementing the two LPbased formulations of sparse regression was written using the tools and solvers present in the commercial software MATLAB [45]. Software is available in "Additional file 1" named as "LPSLGN.tar". It should be straightforward to develop an implementation using C and R wrapper functions for lpsolve [46], a freely available solver for linear, integer and mixed integer programs. The outcome of regression analysis is an optimal weight vector w. Limitations in the numerical precision of solvers means that an element is never exactly zero but a small finite number. Once a solver finds a vector w, a "small" userdefined threshold is used to assign zero and nonzero elements. If the value produced by a solver is greater than the threshold w_{ j }= 1, otherwise w_{ j }= 0. Here, a cutoff of 10^{8} was used.
The computational experiments described here were performed on a large shared machine. The hardware specifications are 6 × COMPAQ AlphaServers ES40 with 4 CPUs per server with 667 MHz, 64 KB + 64 KB primary cache per CPU, 8 MB secondary cache per CPU, 8 GB memory with 4 way interleaving, 4 * 36 GB 10 K rpm Ultra3 SCSI disk drive, and 2*10/100 Mbit PCI Ethernet Adapter. However, the programs can be run readily on a powerful PC. For the MATLAB implementation of the LP formulation based on the general class of linear functions, the LP took a few seconds of wall clock time. An additional few seconds were required to read in files and to set up the problem.
Results and discussion
DREAM2 InSilicoNetwork Challenges data
Statistical assessment of LPSLGNs estimated from simulated data
LPSLGNs based on the general class of linear functions were estimated using the parameter A = 1. For the IN SILICO 1 data set, the Sparsity is ~10. For the IN SILICO 2 data set, the Sparsity is ~13. For the IN SILICO 3 data set, the Sparsity is ~35.
Comparison of the networks – undirected graphs – produced by three different approaches: the LPbased method proposed here, and techniques proposed by the top two teams of the DREAM2 competition (Challenge 4).
Dataset  Team  Precision at k^{ th }correct prediction  Area Under PR Curve  Area Under ROC Curve  

k = 1  k = 2  k = 5  k = 20  
IN SILICO 1  Team 70  1.000000  1.000000  1.000000  1.000000  0.596721  0.829266 
Team 80  0.142857  0.181818  0.045045  0.059524  0.070330  0.459704  
LPSLGN  0.083333  0.086957  0.089286  0.117647  0.087302  0.509624  
IN SILICO 2  Team 80  0.333333  0.074074  0.102041  0.069204  0.080266  0.536187 
Team 70  0.142857  0.250000  0.121320  0.081528  0.084303  0.511436  
LPSLGN  1.000000  1.000000  0.192308  0.183486  0.200265  0.750921  
IN SILICO 3  LPSLGN  0.068966  0.068966  0.068966  0.068966  0.068966  0.500000 
S. cerevisae transcript profiling data
Statistical assessment of LPSLGNs estimated from real data
Biological evaluation of S. cerevisiae LPSLGNs
The learned LPSLGNs provide a forum for generating biological hypotheses and thus directions for future experimental investigations. The edge between SWE1 and BEM1 indicates that the transcript levels of these two genes exhibit a linear relationship; the physical interactions section of their SGD [48] entries indicates that the encoded proteins interact. These results suggests that cellular and/or environmental factor(s) that perturb the transcript levels of both SWE1 and BEM1 may affect cell polarity and cell cycle. NCE102 is connected to genes involved in cell cycle regulation (CDC5) and cell wall remodelling (CWP1, MNN1). A recent report indicates that the transcript level of NCE102 changes when S. cerevisiae cells expressing human cytochrome CYP1A2 are treated with the hepatotoxin and hepatocarcinogen aflatoxin B1 [52]. Thus, this uncharacterized gene may be part of a cell cyclerelated response to genotoxic and/or other stress.
Studies of the yeast NCE102 gene may be relevant to human health and disease. The protein encoded by NCE102 was used as the query for a PSIBLAST [53] search using the WWW interface to the software at NCBI and default parameter settings. Amongst the proteins exhibiting statistically significant similarity (Evalue ≪ 1e  05) were members of the mammalian physin and gyrin families, fourtransmembrane domain proteins with roles in vesicle trafficking and membrane morphogenesis [54]. Human synaptogyrin 1 (SYNGR1; Evalue ~ 1e  28) has been linked to schizophrenia and bipolar disorder [55].
Conclusion
Like this work, a previous study [17] framed the question of deducing the structure of a genetic network from transcript profiling data as a problem of sparse linear regression. The earlier investigation utilized SVD and robust regression to deduce the structure of a network. In particular, the set of all possible networks was characterized by a connectivity matrix A defined by the equation A = A_{0} + CV^{⊤}. The matrix A_{0} computed from the data matrix E via SVD can be seen as the best, in the l_{2} norm sense, connectivity matrix which can generate the data. The matrix V is the right singular vectors of E. The requirement of a sparse graph was enforced by choosing the matrix C such that most of the entries in the matrix A are zero. An approximate solution to the original equation was obtained by posing it as a robust regression problem such that CV^{⊤} = A_{0} was enforced approximately. This new regression problem was solved by formulating an LP that included an l_{1} norm penalty for deviations from equality. In contrast, the solution to the sparse linear regression problem proposed here avoids the need for SVD by formulating the problem directly within the framework of LOO Error and Empirical Risk Minimization and enforcing sparsity via an upper bound on the l_{1} norm of the weight vector, i.e., the original regression problem is posed as a series of LPs. The virtues of this LPbased approach for learning the structure of SLGNs include (i) the method is tractable, (ii) a sparse graph is produced because very few predictor variables are used, (iii) the network model can be parametrized by a positive class of linear functions to produce LPs with few variables, (iv) efficient algorithms and resources for solving LPs in many thousands of variables and constraints are widely and freely available, and (v) the learned network models are biologically reasonable and can be used to devise hypotheses for subsequent experimental investigation.
Another method for deducing the structure of genetic networks framed the task as one of finding a sparse inverse covariance matrix from a sample covariance matrix [56]. This approach involved solving a maximum likelihood problem with an l_{1}norm penalty term added to encourage sparsity in the inverse covariance matrix. The algorithms proposed for this can do no better than O(N^{3}). Better results were achieved by incorporating prior information about error in the sample covariance matrix. In contrast, the LPbased approach to the sparse linear regression problem avoids calculation of a covariance matrix and does not require prior knowledge. Furthermore, the approach proposed here can learn networks with thousands genes in a few minutes on a personal computer.
The quality and utility of the learned LPSLGNs could be enhanced in a number of ways. The network models examined here were estimated from transcript profiles that were subject to minimal data preprocessing. Appropriate lowlevel analysis of profiling data is known to be important [57] so estimating network models from suitably processed data would improve both their accuracy and reliability. The biological predictions were made by visual inspection of a small portion of the LPSLGNs and in an adhoc manner. Hypotheses could be generated in a systematic manner by exploiting statistical and topological properties of sparse undirected graphs. For example, a feature that unites the local and global aspects of a node is its "betweenness", the influence the node has over the spread of information through the graph. The randomwalk betweenness centrality of a node [58] captures the proportion of times a node lies on the path between other nodes in the graph. Nodes with high betweenness but small degree (low connectivity) are likely to play a role in maintaining the integrity of the graph. Betweenness values could be computed from a weighted undirected graph created from an ensemble of LPSLGNs produced by varying the userdefined parameter A. Given a variety of LPSLGNs estimated from data, the cost of an edge could be equated with the frequency with it appears in the learned network models. For the profiling data analyzed here, genes with high betweenness and low degree may have important but unrecognized roles in the S. cerevisae cell cycle and hence correspond to good candidates for experimental investigations of this phenomenon.
The weighted sparse undirected graph described above could serve as the starting point for integrated computational – experimental studies aimed at learning the topology and probability parameters of a probabilistic directed graphical model, a more realistic representation of a genetic network because the edges are oriented and the statistical framework provides powerful tools for asking questions related to the values of variables (nodes) given the values of other variables (inference), handling hidden or unobserved variables, and so on. However, estimating the topology of probabilistic directed graphical model representations of genetic networks from transcript profiling data is challenging [59]. Genes with high betweenness and low degree could be targeted for intervention studies whereby a specific gene would be knocked out in order to determine the orientation of edges associated with it (see, for example, [60]). A variety of theoretical improvements are possible. An explicit model for uncertainty in transcript profiling data could be used to formulate and then solve robust sparse linear regression problems and hence produce models of genetic networks that are more resilient to variation in training data than those generated using the Huber loss function considered here. Expanding the class of interactions from linear models to nonlinear models is an important research topic.
Note
Declarations
Acknowledgements
ISM was supported by grants from the U.S. National Institute on Aging and U.S. Department of Energy (OBER). CB and NC are supported by a grant from MHRD, Government of India.
Authors’ Affiliations
References
 GEO. http://www.ncbi.nlm.nih.gov/geo/
 ArrayExpress. http://www.ebi.ac.uk/arrayexpress/
 Arnone MI, Davidson EH: Hardwiring of Development: Organization and function of Genomic Regulatory Systems. Development. 1997, 124: 18511864.PubMedGoogle Scholar
 Guelzim N, Bottani S, Bourgine P, Képès F: Topological and causal structure of the yeast transcriptional regulatory network. Nature Genetics. 2002, 31: 6063.PubMedView ArticleGoogle Scholar
 Luscombe NM, Babu MM, Yu H, Snyder M, Teichmann SA, Gerstein M: Genomic analysis of regulatory network dynamics reveals large topological changes. Nature. 2004, 431: 308312.PubMedView ArticleGoogle Scholar
 Jordan M: Graphical models. Statistical Science. 2004, 19: 140155.View ArticleGoogle Scholar
 Spirtes P, Glymour C, Scheines R, Kauffman S, Aimale V, Wimberly F: Constructing Bayesian Network models of gene expression networks from microarray data. Proceedings of the Atlantic Symposium on Computational Biology, Genome Information Systems & Technology. 2000Google Scholar
 Jong HD: Modeling and Simulation of Genetic Regulatory Systems: A Literature review. Journal of Computational Biology. 2002, 9: 67103.PubMedView ArticleGoogle Scholar
 Wessels LFA, Someren EPA, Reinders MJT: A comparison of genetic network models. Pacific Symposium on Biocomputing '01. 2001, 6: 508519.Google Scholar
 Andrecut M, Kauffman SA: A simple method for reverse engineering causal networks. PubMed Journal of Physics A: Mathematical and General(46).Google Scholar
 Liang S, Fuhrman S, Somogyi R: Reveal, a general reverse engineering algorithm for inference of genetic network architectures. Pac Symp Biocomput. 1998, 1829.Google Scholar
 Akutsu T, Miyano S, Kuhara S: Identification of genetic networks from a small number of gene expression patterns under the Boolean network model. Pacific Symposium on Biocomputing. 1999, 4: 1728.Google Scholar
 Shmulevich I, Dougherty E, Kim S, Zhang W: Probabilistic Boolean Networks: a rulebased uncertainty model for gene regulatory networks. Bioinformatics. 2002, 18: 261274.PubMedView ArticleGoogle Scholar
 Friedman N, Yakhini Z: On the sample complexity of learning Bayesian networks. PubMed Conference on Uncertainty in Artificial Intelligence. 1996, 272282.Google Scholar
 D'Haeseleer P, Wen X, Fuhrman S, Somogyi R: Linear modelling of mrna expression levels during cns development and injury. Pacific Symposium on Biocomputing '99. 1999, 4: 4152.Google Scholar
 Someren E, Wessels LFA, Reinders M: Linear Modelling of genetic networks from experimental data. Proceedings of the eighth international conference on Intelligent Systems for Molecular Biology. 2000, 355366.Google Scholar
 Yeung M, Tegnér J, Collins J: Reverse engineering gene networks using singular value decomposition and robust regression. Proc Natl Acad Sci USA. 2002, 99: 61636168.PubMedPubMed CentralView ArticleGoogle Scholar
 Stolovitzky G, Monroe D, Califano A: Dialogue on ReverseEngineering Assessment and Methods: The DREAM of HighThroughput Pathway Inference. Annals of the New York Academy of Sciences. 2007, 1115: 122.PubMedView ArticleGoogle Scholar
 Weaver D, Workman C, Stormo G: Modelling regulatory networks with weight matrices. Pacific Symposium on Biocomputing '99. 1999, 4: 112123.Google Scholar
 Chen T, He H, Church G: Modelling gene expression with differential equations. Pacific Symposium on Biocomputing '99. 1999, 4: 2940.Google Scholar
 Butte A, Tamayo P, Slonim D, Golub T, Kohane I: Discovering functional relationships between RNA expression and chemotherapeutic susceptibility using relevance networks. Proc Natl Acad Sci USA. 2000, 97: 1218212186.PubMedPubMed CentralView ArticleGoogle Scholar
 Basso K, Margolin A, Stolovitzky G, Klein U, DallaFavera R, Califano A: Reverse engineering of regulatory networks in human B cells. Nature Genetics. 2005, 37: 382390.PubMedView ArticleGoogle Scholar
 Margolin AA, Nemenman I, Basso K, Wiggins C, Stolovitzky G, Dalla Favera R, Califano A: ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinformatics. BMC Bioinformatics. 2006, 7 (Suppl 1):Google Scholar
 Schäfer J, Strimmer K: An empirical Bayes approach to inferring largescale gene association networks. Bioinformatics. 2005, 21: 754764.PubMedView ArticleGoogle Scholar
 Friedman N: Inferring Cellular Networks Using Probabilistic Graphical Models. Science. 2004, 303 (5659): 799805.PubMedView ArticleGoogle Scholar
 Andrecut M, Kauffman SA: On the sparse reconstruction of gene networks. PubMed Journal of computational biology.Google Scholar
 Andrecut M, Huang S, Kauffman SA: Heuristic Approach to Sparse Approximation of Gene Regulatory Networks. Journal of Computational Biology. 2008, 15 (9): 11731186.PubMedView ArticleGoogle Scholar
 Akutsu T, Kuhara S, Maruyama O, Miyano S: Identification of Gene Regulatory Networks by Strategic Gene Disruptions and Gene Overexpressions. SODA. 1998, 695702.Google Scholar
 Murphy K, Mian I: Modelling gene expression data using Dynamic Bayesian Networks. 1999, Tech. rep., Division of Computer Science, University of California Berkeley, http://www.cs.berkeley.edu/~murphyk/Papers/ismb99.ps.gzGoogle Scholar
 Murphy K: Learning Bayes net structure from sparse data sets. 2001, Tech. rep., Division of Computer Science, University of California Berkeley, http://http.cs.berkeley.edu/~murphyk/Papers/bayesBNlearn.ps.gzGoogle Scholar
 Friedman N, Linial M, Nachman I, Pe'er D: Using Bayesian Networks to Analyze Expression Data. Journal of Computational Biology. 2000, 7: 601620.PubMedView ArticleGoogle Scholar
 Imoto S, Kim S, Goto T, Aburatani S, Tashiro K, Kuhara S, Miyano S: Bayesian Networks and Heteroscedastic for nonlinear modelling of Genetic Networks. Computer Society Bioinformatics Conference. 2002, 219227.View ArticleGoogle Scholar
 Hartemink A, Gifford D, Jaakkola T, Young R: Using Graphical Models and Genomic Expression Data to Statistically Validate Models of Genetic Regulatory Networks. Pacific Symposium on Biocomputing 2001 (PSB01). Edited by: Altman R, Dunker A, Hunter L, Lauderdale K, Klein T. 2001, 422433. New Jersey: World ScientificGoogle Scholar
 Tibshirani R: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B. 267288.Google Scholar
 Kaern M, Elston T, Blake W, Collins J: Stochasticity in gene expression: from theories to phenotypes. Nature Review Genetics. 2005, 6: 451464.View ArticleGoogle Scholar
 DREAM Project. http://wiki.c2b2.columbia.edu/dream/index.php/The_DREAM_Project/DREAM2_Data
 Eisen M, Spellman P, Brown P, Bottstein D: Cluster Analysis and display of genomewide expression patterns. Proceedings of the National Academy of Sciences of the USA. 1998, 95: 1486314868.PubMedPubMed CentralView ArticleGoogle Scholar
 Scoring Methodologies for DREAM2. http://wiki.c2b2.columbia.edu/dream/data/goldstandards/Scoring_Methodologies_for_DREAM2.doc
 Amaldi E, Kann V: On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science. 1998Google Scholar
 Chen SS, Donoho DL, Saunders MA: Atomic Decomposition by Basis Pursuit. 1996, Tech. Rep. Dept. of Statistics Technical Report, Stanford UniversityGoogle Scholar
 Donoho DL, Elad M, Temlyakov V: Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans Inform Theory. 2004, 52: 618.View ArticleGoogle Scholar
 Weston J, Elisseff A, Schölkopf B, Tipping M: Use of the ZeroNorm with Linear Models and Kernel Methods. Journal of Machine Learning Research. 2003, 3:Google Scholar
 McDiarmid C: On the method of bounded differences. Survey in Combinatorics. 1989, 148188. Cambridge University PressGoogle Scholar
 Bousquet O, Elisseeff A: Stability and Generalization. 2000, Tech. rep., Centre de Mathematiques AppliqueesGoogle Scholar
 MATLAB. http://www.mathworks.com/products/matlab/
 Lpsolve. http://packages.debian.org/stable/math/lpsolve
 Newman M: The physics of Networks. Physics Today. 2008Google Scholar
 SGD. http://www.yeastgenome.org/
 GO. http://www.geneontology.org/
 InterPro. http://www.ebi.ac.uk/interpro/
 KEGG. http://www.genome.jp/kegg/pathway.html
 Guo Y, Breeden L, Fan W, Zhao L, Eaton D, Zarbl H: Analysis of cellular responses to aflatoxin B(1) in yeast expressing human cytochrome P450 1A2 using cDNA microarrays. Mutat Res. 2006, 593: 121142.PubMedView ArticleGoogle Scholar
 BLAST. http://www.ncbi.nlm.nih.gov/Education/BLASTinfo/information3.html
 Hubner K, Windoffer R, Hutter H, Leube R: Tetraspan vesicle membrane proteins: synthesis, subcellular localization, and functional properties. Int Rev Cytol. 2002, 214: 103159.PubMedView ArticleGoogle Scholar
 Verma R, Kubendran S, Das SSK, Jain , Brahmachari S: SYNGR1 is associated with schizophrenia and bipolar disorder in southern India. J Hum Genet. 2005, 50: 635640.PubMedView ArticleGoogle Scholar
 Banerjee O, Ghaoui LE, d'Aspremont A, Natsoulis G: Convex optimization techniques for fitting sparse Gaussian graphical models. ICML '06. 2006, 8996.View ArticleGoogle Scholar
 Rubinstein B, McAuliffe J, Cawley S, Palaniswami M, Ramamohanarao K, Speed T: Machine Learning in LowLevel Microarray Analysis. SIGKDD Explorations. 2003, 5:Google Scholar
 Newman M: A measure of betweenness centrality based on random walks. PubMed. 2003, http://aps.arxiv.org/abs/condmat/0309045/Google Scholar
 Friedman N, Koller D: Being Bayesian about network structure: a Bayesian approach to structure discovery in Bayesian Networks. Machine Learning. 2003, 50: 95126.View ArticleGoogle Scholar
 Sachs K, Perez O, Peér D, Lauffenburger D, Nolan G: Causal proteinsignaling networks derived from multiparameter singlecell data. Science. 2005, 308: 523529.PubMedView ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.