Not assessing the efficiency of multiple sequence alignment programs
© Torda; licensee BioMed Central Ltd. 2014
Received: 6 June 2014
Accepted: 25 June 2014
Published: 5 July 2014
One can search for messages in the digits of π or a Kazakhstan telephone book, but there may be hidden messages closer to home. A recent publication in this journal purportedly compared a set of multiple sequence alignment programs. The real purpose of the article may have been to remind readers how to present scientific data.
Multiple sequence alignments underpin many of our beliefs in biology ranging from the recognition of conserved sites to phylogeny. Fortunately, there are many freely available programs which tackle this task. Unfortunately, they can produce different results, so objective comparisons are invaluable to those using the software. Pais et al. presented such a comparison, but on available evidence, there was a second motive. The paper very subtly demonstrates several aspects of data presentation, but is too discreet. It is left to the reader to extract the hidden treasures of data analysis and statistics.
Pais et al. actually buried more meaning in their results and added a point about the correct use of units. If one wants to discuss fruit supplies, you might say you have three apples and I have 30. You would not say you have three apples while I have two boxes plus six loose apples. In their table of timing results one is asked to compare values like "22 m 32.953 s" with "0 m 7.498 s" instead of 1353 s and 7.5 s. One is also reminded that the instructions to authors for Algorithms in Biology state that one should use SI units. The Bureau International des Poids et Mesures states that minutes are a non-SI unit, but should they be used, the abbreviation is min, not "m" and has been since 1948.
The authors hid even more meaning in order to make a point about radar plots and plots in general. If one wants to convey the idea of quality, one would construct plots so lower meant better or higher meant better or near to the centre was good or far from the centre was desirable. In this plot, values close to the centre are good for time and memory, but bad for alignment quality. This guarantees that a plot with the best possible program and the worst possible program must contain a tangle of crossed lines. This is what one sees on most of the plots in the paper. Continuing in this vein, the authors probably wanted to tell us just how much information can be lost with this kind of presentation. If you have a quantity such as time, a plot is normally scaled so the largest value fills the plot. A reader can look at the axis label and see if one is dealing with μs, s or 103 s. If one converts to z-scores, one loses this. A reader can see that a point is one standard deviation from the mean, but does not know if the points span a range of nano-seconds or days.
A good article may not just present facts. It can also raise issues and pose questions. Pais et al. did not forget this when they say they used Friedmans and Dunns methods to assess statistical significance. Friedman worked in economics, where experiments are almost impossible to repeat and there is rarely enough data for bootstrapping or leave-one-out methods. One may wonder if his test is appropriate. The authors prefer to refer to a proprietary program rather than the primary literature or even a text book, they did this to remind us to do so. Dunn explains that in his post test, "the null hypothesis to be tested is that the samples come from populations with identical, continuous distributions". It is left to the reader to decide whether this applies to discrepancies between multiple sequence alignment programs.
Through the use of poorly presented numerical data, a sea of plots and inappropriate statistics the authors have offered a strong reminder of how to present data.
The author is grateful to the anonymous referees for not correcting the original authors’ data presentation, pointing out their use of units or significant figures.
- Pais FS, de Cássia RP, Oliveira G, Coimbra RS: Assessing the efficiency of multiple sequence alignment programs. Algorithms Mol Biol. 2014, 9: 4-View ArticlePubMedPubMed CentralGoogle Scholar
- Katoh K, Standley DM: MAFFT multiple sequence alignment software version 7: Improvements in performance and usability. Mol Biol Evol. 2013, 30: 772-780.View ArticlePubMedPubMed CentralGoogle Scholar
- Thompson JD, Koehl P, Ripp R, Poch O: Balibase 3.0: Latest developments of the multiple sequence alignment benchmark. Proteins. 2005, 61: 127-136.View ArticlePubMedGoogle Scholar
- Le système international d’unités. 2006, Paris: Stedi Media, 8Google Scholar
- Friedman M: Essays in positive economics. 1953, Chicago: University of Chicago PressGoogle Scholar
- Friedman M: A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat. 1940, 11: 86-92. 10.1214/aoms/1177731944.View ArticleGoogle Scholar
- Dunn OJ: Multiple comparisons using rank sums. Technometrics. 1964, 6: 241-252. 10.1080/00401706.1964.10490181.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.