 Research
 Open Access
 Published:
An enhancement of binary particle swarm optimization for gene selection in classifying cancer classes
Algorithms for Molecular Biology volume 8, Article number: 15 (2013)
Abstract
Background
Gene expression data could likely be a momentous help in the progress of proficient cancer diagnoses and classification platforms. Lately, many researchers analyze gene expression data using diverse computational intelligence methods, for selecting a small subset of informative genes from the data for cancer classification. Many computational methods face difficulties in selecting small subsets due to the small number of samples compared to the huge number of genes (highdimension), irrelevant genes, and noisy genes.
Methods
We propose an enhanced binary particle swarm optimization to perform the selection of small subsets of informative genes which is significant for cancer classification. Particle speed, rule, and modified sigmoid function are introduced in this proposed method to increase the probability of the bits in a particle’s position to be zero. The method was empirically applied to a suite of ten wellknown benchmark gene expression data sets.
Results
The performance of the proposed method proved to be superior to other previous related works, including the conventional version of binary particle swarm optimization (BPSO) in terms of classification accuracy and the number of selected genes. The proposed method also requires lower computational time compared to BPSO.
Background
Scientists are allowed to measure the expression levels of thousands of genes simultaneously in the field of biological organisms by utilizing recent advancement in microarrays technology. This technology allows and produces databases of cancerous tissues based on gene expression data [1]. The output of the microarrays technology are gene expression data which consist of valuable information of genomic, diagnostic, and prognostic for researchers. Thus, it is necessary to select informative genes that are really related to cancerous state [2]. However, the gene selection process normally faces problems due to the subsequent features of gene expression data: the large number of genes compared to the small number of samples (highdimensional data), irrelevant genes, and noisy data. Gene selection is called feature selection in artificial intelligence domain. The gene selection has several advantages [3]:

1)
keeps or enhances classification accuracy.

2)
minimizes dimensionality of the data.

3)
reduces computational time.

4)
eliminates irrelevant and noisy genes.

5)
minimizes the cost in a clinical setting.
Gene expression analysis has been intensively researched for more than a decade [4]. Selecting a smaller subset of informative genes from thousands of genes is a crucial step for accurate cancer classification based on gene expression data. Gene selection methods can be classified into two categories in the context of machine learning [3]. A gene selection method is categorized as a filter method if it is carried out independently from a classification procedure. Otherwise, it is called as a hybrid (wrapper) method. Most previous works in the early era of microarrays analysis have used the filter method to select genes due to its lower computation compared to the hybrid method. Ttest, signaltonoiseratio, information gain, etc. are the examples of the filter method that are usually considered as individual generanking methods. Researchers evaluate a gene based on its discriminative power for the target classes without considering its correlations with other genes, which may result in the inclusion of irrelevant and noisy genes in a gene subset for cancer classification. These genes increase the dimensionality of the gene subset and, in turn affect the classification performance. The filter methods also select a number of genes manually, which is known having difficulties in usage, especially for beginner biologists. It has been demonstrated that genes in a cell do not act independently. They interact with one another to complete certain biological processes or to implement certain molecular functions [5].
Therefore, several hybrid methods, especially combinations between particle swarm optimization (PSO) and a classifier, have been implemented to select informative genes in since few years ago up to recent time [3, 6–9]. The hybrid methods usually produce better result in term of accuracy than the filter methods, since the genes are selected by considering and optimizing the correlations among genes. PSO is a new population based stochastic optimization technique. It was proposed by Kennedy and Eberhart [10] based on the observation of social behavior of organisms such as bird flocking and fish schooling. PSO has been successfully implemented in many researches and application areas. It also has only few parameters to adjust and is easier to apply in many applications.
Shen et al. [3] introduced a hybrid of PSO and tabu search approaches for gene selection. Unfortunately, the results obtained by using the hybrid method were less meaningful because the tabu approaches in PSO were unable to search for a nearoptimal solution in search spaces. Next, Chuang et al. proposed an improved binary PSO [6]. Using the proposed method yields 100% classification accuracy in many data sets, but it utilizes a large number of selected genes (large gene subset) to obtain the high accuracy. This method uses a large number of genes because the global best particle is reset to the zero position when its fitness values do not change after three consecutive iterations. Li et al. brought out an idea of hybrid of PSO and genetic algorithms (GA) for the same purpose thereafter [7]. Unfortunately, the result still did not pose high accuracy and many genes had to be selected for cancer classification because there were no direct probability relations between GA and PSO. Chuang et al. [8, 9] also introduced a combination of tabu search and PSO for gene selection, and currently they proposed a hybrid of BPSO and a combat GA for the same purpose. However, the both proposed approaches [8, 9] still need a high number of selected genes to result high classification accuracy. A significant weakness is also found because of a combination of PSO and a tabu search technique, and a hybrid of BPSO and a combat GA which do not share probability significance in their processes. Generally, the PSObased methods [3, 6–9] are incapable to yield a small (nearoptimal) subset of informative genes for high classification accuracy. This drawback is known caused mainly by the large number of gene (highdimension data).
Accurate classification of microarray data is critical for successful clinical diagnosis and treatment [11]. Thus, one diagnostic goal aims to develop a medical procedure based on the least number of genes that are needed to detect diseases accurately. We propose an enhanced binary PSO (EPSO) to select a small (nearoptimal) subset of informative genes that is most relevant for cancer classification. To test the effectiveness of our proposed method, we applied the EPSO to ten gene expression data sets, including binaryclasses and multiclasses data sets.
This paper is organized as follow. In Section 2, we briefly describe the conventional version of binary PSO and EPSO. Section 3 presents the data sets used and the experimental results. Section 4 summarizes this paper by providing its main conclusions and addresses future directions.
Methods
A conventional version of binary PSO (BPSO)
Binary PSO (BPSO) is initialized with a population of particles. All particles move in a problem space to find the optimal solution for each of the iteration. A potential solution in an ndimensional space is presented by a particle. Each particle has its own position and velocity vectors for guiding its movement. The position and velocity vectors of the ith particle in the ndimension can be represented as X_{ i } = (x_{ i }^{1}, x_{ i }^{2},…, x_{ i }^{n}) and V_{ i } = (v_{ i }^{1}, v_{ i }^{2}, …, v_{ i }^{n}), respectively, where x_{ i }^{d} ∈ {0, 1}; i=1,2,..m (m is the total number of particles) and d=1,2,..n (n is the dimension of data) [12].
v_{ i }^{d} is a real number for the dth dimension of the particle i, where the maximum v_{ i }^{d} is V_{max} = (1/3) × n. This maximum value is important, as it controls the granularity of the search by clamping the escalating velocities. A large value of V_{max} assists the global exploration, while a small value encourages the local exploitation. If V_{max} is too small, the swarm may not be able to explore sufficiently beyond locally good regions. In addition, the small value of V_{max} increases the number of time steps to reach an optimum and may become trapped in a local optimum. On the other hand, the possibility of missing a good region becomes higher if values of V_{max} is too large. The particles continue to search in the gainless regions of the search space since they may escape from the good solutions. In our preliminary experiment, we have found that an appropriate maximum velocity value (1/3) × n was found after a several preliminary runs. We chose V_{max} = (1/3) × n and limited the velocity within the range [1, (1/3) × n] which prevented an overly large velocity. A particle can be close to an optimal solution, but a high velocity may make it move far away. By limiting the maximum velocity, particles cannot fly too far away from the optimal solution. Therefore, the BPSO method has a greater chance to find the optimal solution under the limit.
For gene selection, a binary bit string of length n represents the vector of particle where n is the total number of genes. Each position vector (X_{ i }) represents a gene subset. The corresponding gene is selected if the value of the bit is 1. On the other hand, the gene is not selected if the value is 0. The following equations show how each particle in the tth iteration updates its own position and velocity:
if Sig(v_{ i }^{d}(t + 1)) > r_{3}^{d}(t), then x_{ i }^{d}(t + 1) = 1; or otherwise
where c_{1} and c_{2} are the acceleration constants in the interval [0,2] and ${r}_{1}^{d}\left(t\right),{r}_{2}^{d}\left(t\right),{r}_{3}^{d}\left(t\right)~U\left(0,1\right)$ are random values in the range [0,1], which are sampled from a uniform distribution. Pbest_{ i }(t) = (pbest_{ i }^{1}(t), pbest_{ i }^{2}(t), …, pbest_{ i }^{n}(t)) and Gbest(t) = (gbest^{1}(t), gbest^{2}(t), …, gbest^{n}(t)) signify the best previous position of the ith particle and the global best position of the swarm (all particles), respectively. They are evaluated by a fitness function. Sig(v_{ i }^{d}(t + 1)) is a sigmoid function where Sig(v_{ i }^{d}(t + 1)) ∈ [0, 1]. w(t) is an inertia weight, which was introduced by Shi and Eberhart [13] as a mechanism to control the exploration and exploitation abilities of the swarm and remove the need for velocity clamping. The momentum of the particle is controlled by the inertia weight through weighting the contribution of the previous velocity, namely controlling how much memory of the previous particle direction will influence the new velocity. A nonlinear decreasing approach was applied in BPSO to update w(t) in each of the iteration. An initially large value decreased nonlinearly to a small value in this approach. It also permitted a shorter exploration time than a linear decreasing approach where more time would be spent on refining solution (exploiting). The w(t) was initialized with a value of 1.4 and was updated as follows [14, 15]:
where MAXITER is the maximum iteration (generation) and Iter(t) is the current iteration. The pseudo code of BPSO described in the following section.
The Pseudo code of BPSO
The pseudo code of BPSO as follows:
Investigating the drawbacks of BPSO and previous PSObased methods
It would be practical to find the limitations of BPSO and previous PSObased methods before attempting to propose EPSO [3, 6–9]. This subsection investigates the limitations of these methods by analyzing Eq. 2 and Eq. 3, which are the most crucial equations for gene selection in binary spaces. Both these equations are also implemented in BPSO and the PSObased methods.
The sigmoid function (Eq. 2) represents a probability for x_{ i }^{d}(t) to be 0 or 1 (P(x_{ i }^{d}(t) = 0) or P(x_{ i }^{d}(t) = 1)). For example,
if v_{ i }^{d}(t) = 0, then Sig(v_{ i }^{d}(t) = 0) = 0.5 and P(x_{ i }^{d}(t) = 0) = 0.5.
if v_{ i }^{d}(t) < 0, then Sig(v_{ i }^{d}(t) < 0) < 0.5 and P(x_{ i }^{d}(t) = 0) > 0.5.
if v_{ i }^{d}(t) > 0, then Sig(v_{ i }^{d}(t) > 0) > 0.5 and P(x_{ i }^{d}(t) = 0) < 0.5.
In addition, P(x_{ i }^{d}(t) = 0) = 1 − P(x_{ i }^{d}(t) = 1). Thus, we concluded that P(x_{ i }^{d}(t) = 0) = P(x_{ i }^{d}(t) = 1) = 0.5 for the initial iteration, because Eq. 2 is a standard sigmoid function without any constraint and no modification. Although the next iterations potentially influence the P(x_{ i }^{d}(t) = 0) or P(x_{ i }^{d}(t) = 1), the P(x_{ i }^{d}(t) = 0) = P(x_{ i }^{d}(t) = 1) = 0.5 mostly maintained its application on the gene expression data because the gene expression data was highly dimensional and had a large search space. It only minimized the number of genes to about half of the total number of genes by using the standard sigmoid function in highdimensional data. This is shown and proven in the section of experimental results. Therefore, Eq. 2 and Eq. 3 could potentially be the shortcomings of BPSO and the previous PSObased methods in selecting a small number of genes for producing a nearoptimal (small) subset of genes from the gene expression data.
An enhanced binary PSO (EPSO)
In almost all previous gene expression data researches, a subset of genes is commonly selected for excellent cancer classifications. Thus, we propose EPSO for the selection of a nearoptimal (small) subset of genes in order to overcome the shortcomings of BPSO and previous PSObased methods [3, 6–9]. EPSO differs from the BPSO and PSObased methods in three ways: 1) we introduced a scalar quantity called particle speed (s); 2) we modified the existing sigmoid function; 3) we proposed a rule for updating x_{ i }^{d}(t + 1). In contrast, the BPSO and PSObased methods do not apply particle speed, use the standard sigmoid function (Eq. 2), and implement the original rule (Eq. 3). The particles’ speed, modified sigmoid function, and new rule were introduced in order to:
maximize the probability of x_{ i }^{d}(t + 1) = 0 (P(x_{ i }^{d}(t + 1) = 0)) and meanwhile
minimize the probability of x_{ i }^{d}(t + 1) = 1 (P(x_{ i }^{d}(t + 1) = 1)).
A small number of genes were selected and grouped into a gene subset caused by the increased and decreased probability values. x_{ i }^{d}(t + 1) = 1 represents that the corresponding gene is selected. Otherwise, x_{ i }^{d}(t + 1) = 0 indicates that the corresponding gene is not selected.
Definition 1. s _{ i } is the speed, length or magnitude of V _{ i } for the particle i. Therefore, the following properties of s _{ i } are crucial:
nonnegativity: s_{ i } ≥ 0;
definiteness: s_{ i } = 0 if and only if V_{ i } = 0;
homogeneity: ‖αV_{ i }‖ = α‖V_{ i }‖ = αs_{ i }where α ≥ 0;
the triangle inequality: ‖V_{ i } + V_{i+1}‖ ≤ ‖V_{ i }‖ + ‖V_{i+1}‖ where ‖V_{ i }‖ = s_{ i }and ‖V_{i+1}‖ = s_{i+1}.
The particles’ speed (Equation 5), sigmoid function (Equation 6), and the rule (Equation 7) are proposed as follow:
subjected to s_{ i }(t + 1) ≥ 0 if Sig(s_{ i }(t + 1)) > r_{3}^{d}(t), then x_{ i }^{d}(t + 1) = 0; or else
where s_{ i }(t + 1) represents the speed of the particle i for the t+1 iteration. By contrast, in BPSO and other PSObased methods (Eq. 1, Eq. 2, and Eq. 3), v_{ i }^{d}(t + 1) is related to a single element of a particle velocity vector for the particle i. In EPSO, Eq. 5, Eq. 6, and Eq. 7 were used to replace Eq. 1, Eq. 2, and Eq. 3, respectively. s_{ i }(t + 1) is the rate at which the particle i changes its position. s_{ i }(t + 1) ≥ 0 is the most significant property of s_{ i }(t + 1) based on Definition 1. Hence, s_{ i }(t + 1) was used instead of v_{ i }^{d}(t + 1) so that its positive value could increase P(x_{ i }^{d}(t + 1) = 0).
s_{ i }(t + 1) for each of the particle was initialized with positive real numbers in Eq. 5. The distance between Pbest_{ i }(t) and X_{ i }(t) (dist(Pbest_{ i }(t) − X_{ i }(t))), and the distance between Gbest(t) and X_{ i }(t) (dist(Gbest(t) − X_{ i }(t))), are the basic features that needed to be taken into consideration for calculating and updating s_{ i }(t + 1) in Eq. 5, whereas v_{ i }^{d}(t + 1) was calculated by using the original formula (Eq. 1) and it was fundamentally based on the difference between Pbest_{ i }^{d}(t) and x_{ i }^{d}(t), and the difference between Gbest^{d}(t) and x_{ i }^{d}(t). The distances were used in the calculation for updating s_{ i }(t + 1) in order to make sure that Eq. 6 always satisfied the property of s_{ i }(t + 1), namely (s_{ i }(t + 1) ≥ 0) and to increase P(x_{ i }^{d}(t + 1) = 0). The next subsection shows how to calculate the distance between two positions of two particles, e.g., dist(Gbest(t) − X_{ i }(t)).
Equations (5–7) and s_{ i }(t) ≥ 0 increase P(x_{ i }^{d}(t) = 0) due to the minimum value for P(x_{ i }^{d}(t) = 0) which was 0.5 when s_{ i }(t) = 0 ( min P(x_{ i }^{d}(t) = 0) ≥ 0.5). In addition, they reduced the maximum value for P(x_{ i }^{d}(t) = 1) to 0.5 ( max P(x_{ i }^{d}(t) = 1) ≤ 0.5). Therefore, if s_{ i }(t) > 0, then P(x_{ i }^{d}(t) = 0) > > 0.5 and P(x_{ i }^{d}(t) = 1) < < 0.5.
Figure 1 shows that a) Equations (5–7) and s_{ i }(t) ≥ 0 in EPSO increase P(x_{ i }^{d}(t) = 0); b) Equations (1–3) in BPSO yield P(x_{ i }^{d}(t) = 0) = P(x_{ i }^{d}(t) = 1) = 0.5. As an example, the calculations for P(x_{ i }^{d}(t) = 0) and P(x_{ i }^{d}(t) = 1) in Figure 1(a) are shown as follow:
if s_{ i }(t) = 1, then P(x_{ i }^{d}(t) = 0) = 0.993307 and P(x_{ i }^{d}(t) = 1) = 1 − P(x_{ i }^{d}(t) = 0) = 0.006693.
if s_{ i }(t) = 2, then P(x_{ i }^{d}(t) = 0) = 0.999955 and P(x_{ i }^{d}(t) = 1) = 1 − P(x_{ i }^{d}(t) = 0) = 0.000045.
From the calculation above (based on Eq. 6), the modified sigmoid function (Eq. 6) generates higher P(x_{ i }^{d}(t) = 0) compared to the standard sigmoid function (Eq. 2). For example, the calculations for P(x_{ i }^{d}(t) = 0) and P(x_{ i }^{d}(t) = 1) based on Eq. 2 in Figure 1(b) are shown as follow:
if v_{ i }^{d}(t) = 1, then P(x_{ i }^{d}(t) = 0) = 0.731059 and P(x_{ i }^{d}(t) = 1) = 1 − P(x_{ i }^{d}(t) = 0) = 0.268941.
if v_{ i }^{d}(t) = 2, then P(x_{ i }^{d}(t) = 0) = 0.880797 and P(x_{ i }^{d}(t) = 1) = 1 − P(x_{ i }^{d}(t) = 0) = 0.119203.
A small number of genes were selected in order to produce a nearoptimal (small) gene subset from highdimensional data (gene expression data) based on the high probability of x_{ i }^{d}(t) = 0 (P(x_{ i }^{d}(t) = 0)). Hence, EPSO is proposed to surmount the limitations of BPSO and the previous PSObased methods and to produce a small subset of informative genes as a result.
Calculating the distance of two particles’ positions
The number of different bits between two particles is related to the difference between their positions. For example, let us consider Gbest(t) = [0, 0, 1, 1, 1, 0, 1, 0, 0, 0] and X_{ i }(t) = [1, 1, 1, 0, 1, 1, 0, 1, 0, 0]. The difference between Gbest(t) and X_{ i }(t) is indicated as diff(Gbest(t) − X_{ i }(t)) = [ − 1, − 1, 0, 1, 0, − 1, 1, − 1, 0, 0]. A value of 1 indicates that this bit (gene) should be selected in comparison with the best position. However, if it is not selected, the classification quality may decrease and direct to a lower fitness value. In contrast, a value of −1 shows that this bit should not be selected in comparison with the best position, but it is nonetheless selected. The selection of irrelevant genes makes the length of the subset longer and leads to a lower fitness value. The number of 1 is assumed to be a, whereas the number of −1 is b. We used the absolute value of a − b (a − b) to express the distance between the two positions. In this example, the distance between Gbest(t) and X_{ i }(t) is dist(Gbest(t) − X_{ i }(t)) = a − b = 2 − 4 = 2.
Fitness functions
The fitness value of a particle (a gene subset) is calculated as follows:
in which A(X_{ i }) ∈ [0, 1] is the leaveoneoutcrossvalidation (LOOCV) classification accuracy that uses the only gene in a gene subset (X_{ i }). Support vector machine classifiers (SVM) provide this accuracy. R(X_{ i }) is the number of selected genes in X_{ i }. n is the total number of genes for each sample. The importance of accuracy and the number of selected genes correspond to two priority weights w_{1} and w_{2}, respectively. For this article, the accuracy is more important than the number of selected genes. Thus, we selected the value of w_{1} in the range [0.6, 0.9] and we set w_{2} = 1 − w_{1}. In order to produce the remaining percentage of weights after the value of w_{1} had been chosen, the value of w_{2} was set to 1 − w_{1}.
Results and discussion
Data sets and experimental setup
Table 1 summarizes the gene expression data sets used in this study. The data sets incorporated binaryclasses and multiclasses data sets that have thousands of genes (highdimensional data). They were downloaded from http://www.gemssystem.org. All of the experimental results reported in this article were acquired using Rocks Linux version 4.2.1 (Cydonia) on the IBM xSeries 335 (a cluster machine that contains 13 compute nodes). Each compute node has four high performances 3.0 GHz Intel Xeon CPUs with 512 MB of memory. Thus, a total of 52 CPUs for the 13 compute nodes were used. The IBM xSeries 335 with 52 CPUs was needed to experiment EPSO and BPSO because both these methods have huge computational times and run on highdimensional data. The computational power of IBM xSeries 335 can reduce the computational time of highdimensional data for both methods. Each run had been independently experimented using only one CPU in order to make sure the computational time of every run utilize the same capacity of CPUs usage. This situation was crucial for the performance comparison of computational times between EPSO and BPSO.
Experimental results that were generated by EPSO were evaluated with an experimental method (BPSO) and other previous PSObased methods for objective comparisons [3, 6–9]. Firstly, a gain ratio technique was applied for preprocessing in order to preselect 500topranked genes. This technique is a filterbased feature ranking approach and based on the concept of entropy. The original work of the gain ratio technique can be found in Quinlann [17]. The genes were then applied in EPSO and BPSO. Next, the LOOCV classification accuracy on gene subsets that were produced by EPSO and BPSO, was measured by using SVM. Generally, in LOOCV classification accuracy, N1 samples is trained and tested on the remaining sample which has not been used during the training. N represents the total number of samples. By cycling through all the samples, we can get realistic and honest estimates of the both approaches. The implementation of LOOCV was in exactly the same way as done by Chuang et al. in order to avoid selection bias [6, 8, 9] where the only one crossvalidation cycle (outer loop); namely LOOCV, was utilized, instead of two nested ones. Several experiments were independently carried out 10 times on each data set using EPSO and BPSO. Next, an average result of the 10 independent runs was achieved. To evaluate the performances of EPSO and BPSO, three criteria, following their importance, were considered: LOOCV classification accuracy, the number of selected genes, and computational times. Each criterion should have the best, average, and standard deviation results. High accuracy, the small number of selected genes, and low computational time were needed to obtain an effective performance.
The parameter values for EPSO and BPSO are shown in Table 2. The results of preliminary runs decided the parameter values. The numbers of particles and iterations to reach a good solution are problem dependent [16]. If the numbers were large, both methods would need more time to complete their processes. If the numbers were small, they would take short period of time, but they would not be able to find a good solution. Therefore, we chose intermediate values for the number of particles and iterations between 100 and 500. The value of w_{1} was larger than w_{2} because the classification accuracy was more important than the number of selected genes. We tried to get the best value based on trial and error approaches. EPSO and BPSO were analyzed using different parameter values. So far, the best values for both w_{1} and w_{2} using these methods were 0.8 and 0.2, respectively. When w_{2} was more than w_{1}, the number of selected genes also increased. c_{1} and c_{2} had the same value (2) so that particles would be attracted towards the averages of Pbest_{ i }(t) and Gbest(t) [15].
In the fitness functions of EPSO and BPSO, SVM classifiers have been used to evaluate the subsets of selected genes. A radial basis function (RBF) kernel was implemented on the SVM classifiers. The rationale to use this kernel was that early results from previous works were readily found with excellent generalization performance in nonlinear separable and low computational cost [18]. This RBF kernel nonlinearly maps samples into a higher dimensional space so it, unlike the linear kernel, can handle the case when the relation between class labels and attributes is nonlinear. For a multiclass SVM, we have applied the “oneagainstone” approach. Hsu and Lin [19] have provided a detailed comparison and conclude that “oneagainstone” is a competitive approach.
Experimental results
The results yielded by EPSO were almost consistent in all data sets based on the standard deviations in Tables 3 and 4. Attractively, 100% LOOCV accuracy with less than 12 selected genes on the Leukemia1, Leukemia2, and DLBCL data sets had been achieved from all runs. Furthermore, EPSO had efficiently selected and yielded a nearoptimal gene subset from highdimensional data (gene expression data), with a proof that over 92% average classification accuracies had been obtained on other data sets, except for the 9_Tumors data set. Moreover, the standard deviations of the number of selected genes were less than 10 for all of the data sets except for the SRBCT data set (13.03 standard deviations). All the best results achieved 100% LOOCV accuracy with not more than 7 selected genes, indicating that EPSO efficiently selected and produced a nearoptimal gene subset from highdimensional data (gene expression data).
Practically, the best subset of a data set is first chosen and the genes in it are then listed for biological usage. These informative genes, among thousands of genes, may be excellent candidates for clinical and medical investigations. Biologists can save time because they can directly refer to the genes that have higher possibilities of being useful for cancer diagnoses and as drug targets in the future. The best subset is chosen based on the highest classification accuracy with the smallest number of selected genes. The highest accuracy provides confidence for the most accurate classification of cancer types. Moreover, the smallest number of selected genes for cancer classification can reduce the cost in clinical settings.
The averages of fitness values of EPSO increased dramatically after a few generations on all the data sets as shown in Figure 2. This trend proved that EPSO is suitable for selecting a small number of genes from highdimensional data (gene expression data) to maximize classification accuracy. It also proved that a combination between a high classification rate and a small number (subset) of selected genes can produce a high fitness value. The condition of the proposed particles’ speed that should always be positive real numbers started in the initialization method, the modified sigmoid function, and the new rule for updating particle’s positions, which provoked the early convergence of EPSO. The fitness value of EPSO still increased for further generations on all the data sets except for 9_Tumors and 11_Tumors data sets. However, there was only a small rise in values. EPSO did the exploration in order to find a good solution and it had been proven by this growth. In contrast, P(x_{ i }^{d}(t) = 0) = P(x_{ i }^{d}(t) = 1) = 0.5. caused the averages of fitness values of BPSO to become lower than the fitness value of EPSO until the last generation, except for the 9_Tumors and 11_Tumors data sets.
In general, it is worthwhile to mention that the classification accuracy, the number of selected genes, and computational times of EPSO were superior to BPSO in terms of the best, average, and standard deviation results on all the data sets except for the 9_Tumors data set according to the result in Table 5. The computational times of BPSO were also higher than EPSO in all the data sets. EPSO was able to minimize its computational times because of the following reasons:

1)
EPSO chose smaller number of genes compared to BPSO;

2)
SVM utilized a small number of features (genes) that were selected by EPSO for the classification process which caused the computation of SVM to be fast;

3)
Only the speed of a particle for comparing with r _{3} ^{d}(t) was used by EPSO, whereas all elements of a particle’s velocity vector for the comparison had been practiced by BPSO.
We compared our work with previous related works that used PSObased methods in their proposed methods for an objective comparison [3, 6–9]. Table 6 shows the result of the comparison. Two criteria were used to evaluate the performance of EPSO and the other methods: classification accuracy and the number of selected genes. Higher accuracy with a smaller number of selected genes is needed to obtain superior performance. For all of the data sets, the averages of the number of the selected genes for our work were smaller than the previous works [3, 6–9]. Our work also produced higher classification accuracies on the six data sets (Brain_Tumor2, Leukemia1, Leukemia2, SRBCT, Prostate_Tumor, and DLBCL) compared to the previous PSObased methods [3, 6–9]. Moreover, our method resulted in higher averages of the classification accuracies on all data sets compared to the other methods [3, 7]. The number of selected genes of our work in terms of average and the best results were also smaller than the previous works for all the data sets. However, Chuang et al. showed better classification accuracies than our work on four data sets (11_Tumors, 9_Tumors, Brain_Tumor1, and Lung_Cancer) [8].
The most recent works came up with similar LOOCV results (100%) with ours on the Leukemia1, Leukemia2, SRBCT, and DLBCL data sets, but they used more than 400 genes to obtain these results [6, 8, 9]. Moreover, their experimental results were obtained by utilizing only one independent run on each data set and not based on average results, causing that they could not have statistically meaningful conclusions. Since the proposed method is a stochastic approach, therefore the average results are important. Overall, our work outperformed the other methods in terms of the LOOCV accuracy and the number of selected genes on six data sets (Brain_Tumor2, Leukemia1, Leukemia2, SRBCT, Prostate_Tumor, and DLBCL). The running times between EPSO and these works could not be compared because they were not reported.
According to Figure 2 and Tables 3–6, EPSO is reliable for gene selection where it had yielded the nearoptimal solution from gene expression data. This was due to the proposed particles’ speed, the modified sigmoid function, and the introduced rule which maximized the probability x_{ i }^{d}(t + 1) = 0 (P(x_{ i }^{d}(t + 1) = 0)). This high probability caused the selection of a small number of informative genes and finally produced a nearoptimal subset (a small subset of informative genes with high classification accuracy) for cancer classification. The particles’ speed was introduced to provide the rate at which a particle changed its position, whereas the rule was proposed to update the particle’s positions. The sigmoid function was modified for increasing the probability of bits in particle’s positions to be zero.
Conclusions
In this paper, EPSO is proposed for gene selection on ten gene expression data sets. In general, the performance of EPSO is better than BPSO and PSObased methods that have been proposed in previous related works, in terms of classification accuracy and the number of selected genes. EPSO is efficient because the probability x_{ i }^{d}(t + 1) = 0 is increased by the particle speed, the modified sigmoid function, and the introduced rule in order to yield a nearoptimal subset of genes for better cancer classification. EPSO also features lower running times because it selects only a smaller number of genes compared to BPSO. A modified representation of particle’s positions in PSO will be proposed to minimize the number of genes subsets in solution spaces for future works
References
 1.
Knudsen S: A biologist’s guide to analysis of DNA microarray data. 2002, New York, USA: John Wiley & Sons
 2.
Mohammad MS, Omatu S, Yoshioka M, Deris S: A cyclic hybrid method to select a smaller subset of informative genes for cancer classification, International Journal of Innovative Computing. Inf Control. 2009, 5 (8): 21892202.
 3.
Shen Q, Shi WM, Kong W: Hybrid particle swarm optimization and tabu search approach for selecting genes for tumor classification using gene expression data. Comput Biol Chem. 2009, 32: 5360.
 4.
Pavlidis SP, Payne AM, Swift SM: Multimembership gene regulation in pathway based microarray analysis. Algorithms for Molecular Biology. 2011, 6: 2210.1186/17487188622.
 5.
Liu Z, Magder LS, Hyslop T, Mao L: Survival associated pathway identification with group L_{P} penalized global AUC maximization. Algorithms for Molecular Biology. 2010, 5: 3010.1186/17487188530.
 6.
Chuang LY, Chang HW, Tu CJ, Yang CH: Improved binary PSO for feature selection using gene expression data. Comput Biol Chem. 2009, 32: 2938.
 7.
Li S, Wu X, Tan M: Gene selection using hybrid particle swarm optimization and genetic algorithm. Soft Computing. 2008, 12: 10391048. 10.1007/s005000070272x.
 8.
Chuang LY, Yang CH, Yang CH: Tabu search and binary particle swarm optimization for feature selection using microarray data. J Comput Biol. 2009, 16 (12): 16891703. 10.1089/cmb.2007.0211.
 9.
Chuang LY, Yang CH, Li JC, Yang CH: A hybrid BPSOCGA approach for gene selection and classification of microarray data. J Comput Biol. 2011, 18: 114. 10.1089/cmb.2010.0102.
 10.
Kennedy J, Eberhart R: Particle swarm optimization. 1995, Perth, Australia: Proc. 1995 IEEE Int. Conf. Neural Networks 4, IEEE Press, 19421948.
 11.
Yao B, Li S, ANMM4CBR: A casebased reasoning method for gene expression data classification. Algorithms For Molecular Biology. 2010, 5: 1410.1186/17487188514.
 12.
Kennedy J, Eberhart R: A discrete binary version of the particle swarm algorithm. Proc. 1997 IEEE Int. Conf. Systems, Man, and Cybernetics, IEEE Press, Florida, USA. 1997, 5: 41044108.
 13.
Shi Y, Eberhart RC: A modified particles swarm optimizer. 1998, Piscataway, NJ: Proc. 1998 IEEE Congress on Evolutionary Computation, IEEE Pres, 6973.
 14.
Naka S, Genji T, Yura T, Fukuyama Y: Practical distribution state estimation using hybrid particle swarm optimization. 2001, Ohio, USA: Proc. 2001 IEEE Power Engineering Society Winter Meeting, IEEE Press, 815820.
 15.
Peram T, Veeramacheneni K, Mohan CK: Fitnessdistanceratio based particle swarm optimization. 2003, Indiana, USA: Proc. 2003 IEEE Swarm Intelligence Symposium, IEEE Press, 174181.
 16.
Engelbrecht AP: Fundamentals of computational swarm intelligence. 2005, West Succex, England: John Wiley and Sons,
 17.
Quinlan JR: Induction of decision tress. Mach Learn. 1986, 1 (1): 81106.
 18.
Suykens JAK, Vandewalle J: Least squares support vector machine classifiers. Neural Processing Letters. 1999, 9 (3): 293300. 10.1023/A:1018628609742.
 19.
Hsu CW, Lin CJ: A comparison of methods for multiclass support vector machines. IEEE Transaction on Neural Networks 2002. 2002, 13 (2): 415425.
Acknowledgements
The authors would like to express gratitude to the referees for helpful suggestions and Universiti Teknologi Malaysia for sponsoring this research with a GUP Research Grant (Vot Number: Q.J130000.7107.01H29).
Author information
Additional information
Competing interests
The authors declared that they have no competing interests.
Authors’ contributions
MSM is a principal researcher and proposed the main idea for this study. SO, SD, MY supported and checked the manuscript. Moreover, they gave well advises. AA and ZI aided in interpretation of the results. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Received
Accepted
Published
DOI
Keywords
 Particle Swarm Optimization
 Classification Accuracy
 Gene Expression Data
 Sigmoid Function
 Gene Subset