Fast algorithms for approximate circular string matching

Background Circular string matching is a problem which naturally arises in many biological contexts. It consists in finding all occurrences of the rotations of a pattern of length m in a text of length n. There exist optimal average-case algorithms for exact circular string matching. Approximate circular string matching is a rather undeveloped area. Results In this article, we present a suboptimal average-case algorithm for exact circular string matching requiring time O(n) . Based on our solution for the exact case, we present two fast average-case algorithms for approximate circular string matching with k-mismatches, under the Hamming distance model, requiring time O(n) for moderate values of k, that is k=O(m/logm) . We show how the same results can be easily obtained under the edit distance model. The presented algorithms are also implemented as library functions. Experimental results demonstrate that the functions provided in this library accelerate the computations by more than three orders of magnitude compared to a naïve approach. Conclusions We present two fast average-case algorithms for approximate circular string matching with k-mismatches; and show that they also perform very well in practice. The importance of our contribution is underlined by the fact that the provided functions may be seamlessly integrated into any biological pipeline. The source code of the library is freely available at http://www.inf.kcl.ac.uk/research/projects/asmf/.


Background
Circular sequences appear in a number of biological contexts. This type of structure occurs in the DNA of viruses [1,2], bacteria [3], eukaryotic cells [4], and archaea [5]. In [6], it was noted that, due to this, algorithms on circular strings may be important in the analysis of organisms with such structure. Circular strings have previously been studied in the context of sequence alignment. In [7], basic algorithms for pairwise and multiple circular sequence alignment were presented. These results were later improved in [8], where an additional preprocessing stage was added to speed up the execution time of the algorithm. In [9], the authors also presented efficient algorithms for finding the optimal alignment and consensus sequence of circular sequences under the Hamming distance metric.
*Correspondence: solon.pissis@kcl.ac.uk 1 King's College London, London, UK Full list of author information is available at the end of the article In order to provide an overview of our results and algorithms, we begin with a few definitions, generally following [10]. We think of a string x of length n as an array x[0. .n − 1], where every x[i], 0 ≤ i < n, is a letter drawn from some fixed alphabet of size σ = | |. The empty string of length 0 is denoted by ε. A string x is a factor of a string y if there exist two strings u and v, such that y = uxv. Let the strings x, y, u, and v be such that y = uxv. If u = ε, then x is a prefix of y. If v = ε, then x is a suffix of y.
Let x be a non-empty string of length n and y be a string. We say that there exists an occurrence of x in y, or, more simply, that x occurs in y, when x is a factor of y. Every occurrence of x can be characterised by a position in y. Thus we say that x occurs at the starting position i in y when y[i. .i + n − 1] = x. The Hamming distance between strings x and y, both of length n, is the number of positions i, 0 ≤ i < n, such that x[i] = y [i]. Given a nonnegative integer k, we write x ≡ k y if the Hamming distance between x and y is at most k. http://www.almob.org/content/9/1/9 A circular string of length n can be viewed as a traditional linear string which has the left-and right-most symbols wrapped around and stuck together in some way. Under this notion, the same circular string can be seen as n different linear strings, which would all be considered equivalent. Given a string x of length n, we denote by x i = x[i. .n − 1] x[ 0. .i − 1], 0 < i < n, the i-th rotation of x and x 0 = x. Consider, for instance, the string x = x 0 = abababbc; this string has the following rotations: x 1 = bababbca, x 2 = ababbcab, x 3 = babbcaba, x 4 = abbcabab, x 5 = bbcababa, x 6 = bcababab, Here we consider the problem of finding occurrences of a pattern string x of length m with circular structure in a text string t of length n with linear structure. For instance, the DNA sequence of many viruses has circular structure, so if a biologist wishes to find occurrences of a particular virus in a carriers DNA sequence-which may not be circular-they must consider how to locate all positions in t that at least one rotation of x occurs. This is the problem of circular string matching.
The problem of exact circular string matching has been considered in [11], where an O(n)-time algorithm was presented. A naïve solution with quadratic complexity consists in applying a classical algorithm for searching a finite set of strings after having built the trie of rotations of x. The approach presented in [11] consists in preprocessing x by constructing a suffix automaton of the string xx, by noting that every rotation of x is a factor of xx. Then, by feeding t into the automaton, the lengths of the longest factors of xx occurring in t can be found by the links followed in the automaton in time O(n). In [12], the authors presented an optimal average-case algorithm for exact circular string matching, by also showing that the average-case lower bound for single string matching of O(n log σ m/m) also holds for circular string matching. Very recently, in [13], the authors presented two fast average-case algorithms based on word-level parallelism. The first algorithm requires average-case time O(n log σ m/w), where w is the number of bits in the computer word. The second one is based on a mixture of word-level parallelism and q-grams. The authors showed that with the addition of q-grams, and by setting q = O(log σ m), an optimal averagecase time of O(n log σ m/m) is achieved. Indexing circular patterns [14] and variations of approximate circular string matching under the edit distance model [15]-both based on the construction of a suffix tree-have also been considered.
In this article, we consider the following problems.
Problem 1 (Exact Circular String Matching). Given a pattern x of length m and a text t of length n > m, find all factors u of t such that u = x i , 0 ≤ i < m.

Problem 2 (Approximate Circular String Matching with k-Mismatches). Given a pattern x of length m, a text t of length n > m, and an integer threshold k < m, find all factors u of t such that u
The aforementioned algorithms for the exact case exhibit the following disadvantages: first, they cannot be applied in a biological context since both single nucleotide polymorphisms as well as errors introduced by wet-lab sequencing platforms might have occurred in the sequences; second, it is not clear whether they could easily be adapted to deal with the approximate case. Similar to the exact case [12], it can be shown that the average-case lower bound for single approximate string matching of O(n(k + log σ m)/m) [16] also holds for approximate circular string matching with kmismatches under the Hamming distance model. To the best of our knowledge, no optimal average-case algorithm exists for this problem. Therefore, to achieve optimality, one could use the optimal average-case algorithm for multiple approximate string matching, presented in [17], for matching the r = m rotations of x requiring, on aver- We show how the same results can be easily obtained under the edit distance model. The presented algorithms are also implemented as library functions. Experimental results demonstrate that the functions provided in this library accelerate the computations by more than three orders of magnitude compared to a naïve approach. The source code of the library is freely available at http://www. inf.kcl.ac.uk/research/projects/asmf/.

Properties of the partitioning technique
In this section, we give a brief outline of the partitioning technique in general; and then show some properties of the version of the technique we use for our algorithms. The partitioning technique, introduced in [18], and in http://www.almob.org/content/9/1/9 some sense earlier in [19], is an algorithm based on filtering out candidate positions that could never give a solution to speed up string-matching algorithms. An important point to note about this technique is that it reduces the search space but does not, by design, verify potential occurrences. To create a string-matching algorithm filtering must be combined with some verification technique. The idea behind the partitioning technique was initially proposed for approximate string matching, but here we show that this can also be used for exact circular string matching.
The idea behind the partitioning technique is to partition the given pattern in such a way that at least one of the fragments must occur exactly in any valid approximate occurrence of the pattern. It is then possible to search for these fragments exactly to give a set of candidate occurrences of the pattern. It is then left to the verification portion of the algorithm to check if these are valid approximate occurrences of the pattern. It has been experimentally shown that this approach yields very good practical performance on large-scale datasets [20], even if it is not theoretically optimal.
For exact circular string matching, for an efficient solution, we cannot simply apply well-known exact stringmatching algorithms, as we must also take into account the rotations of the pattern. We can, however, make use of the partitioning technique and, by choosing an appropriate number of fragments, ensure that at least one fragment must occur in any valid exact occurrence of a rotation. Lemma 1 together with the following fact provide this number.

Fact 1. Any rotation of x
; and any factor of length m of x is a rotation of x. Proof. Immediate from the pigeonhole principle-if n items are put into m < n pigeonholes, then at least one pigeonhole must contain more than one item.
Based on Lemma 2, we take a similar approach to the one described by Lemma 1, to obtain the sufficient number of fragments in the case of approximate circular string matching with k-mismatches. Proof. Let f denote the length of the fragment. If we partition x in 2k + 4 fragments of length Therefore any factor of length m of x , and, by Fact 1, any rotation of x, must contain at least k + 1 of the fragments. For a graphical illustration of this proof inspect Figure 2.

Exact circular string matching via filtering
In this section, we present ECSMF, a new suboptimal average-case algorithm for exact circular string matching via filtering. It is based on the partitioning technique and a series of practical and well-established data structures such as the suffix array (for more details see [21]).

Longest common extension
First, we describe how to compute the longest common extension, denoted by lce, of two suffixes of a string in constant time (for more details see [22]). lce queries are an important part of the algorithms presented later on.
Let SA denote the array of positions of the sorted suffixes of string x of length n, i.e. for all 1 ≤ r < n, we have • Compute arrays SA and iSA of x [21].

Algorithm ECSMF
Given a pattern x of length m and a text t of length n > m, an outline of algorithm ECSMF for solving Problem 1 is as follows.

Construct the string
of length 2m − 1. By Fact 1, any rotation of x is a factor of x . 2. The pattern x is partitioned in 4 fragments of length (2m − 1)/4 and (2m − 1)/4 . By Lemma 1, at least one of the 4 fragments is a factor of any rotation of x. 3. Match the 4 fragments against the text t using an Aho Corasick automaton [25]. Let L be a list of size Occ of tuples, where < p x , , p t >∈ L is a 3-tuple such that 0 ≤ p x < 2m − 1 is the position where the fragment occurs in x , is the length of the corresponding fragment, and 0 ≤ p t < n is the position where the fragment occurs in t. 4. Compute SA, iSA, LCP, and RMQ LCP of T = x t.
Compute SA, iSA, LCP, and RMQ LCP of T r = rev(tx ), that is the reverse string of tx . 5. For each tuple < p x , , p t >∈ L, we try to extend to the right via computing in other words, we compute the length E r of the longest common prefix of x [p x + . . 2m − 1] and t[p t + . . n − 1], both being suffixes of T. Similarly, we try to extend to the left via computing E l using lce queries on the suffixes of T r . 6. For each E l , E r computed for tuple < p x , , p t >∈ L, we report all the valid starting positions in t by first checking if the total length E l + + E r ≥ m; that is the length of the full extension of the fragment is greater than or equal to m, matching at least one rotation of x. If that is the case, then we report positions

Approximate circular string matching with k-mismatches via filtering
In this section, based on the ideas presented in algorithm ECSMF, we present algorithms ACSMF and ACSMF-Simple, two new fast average-case algorithms for approximate circular string matching with k-mismatches via filtering.

Algorithm ACSMF
The first four steps of algorithm ACSMF are essentially the same as in algorithm ECSMF. A small difference exists in Step 2, where the sufficient number of fragments in the case of approximate circular string matching with kmismatches is used. The main difference is in Step 5, where algorithm ACSMF tries to extend k + 1 times to the right and k + 1 times to the left. Given a pattern x of length m, a text t of length n > m, and an integer threshold k < m, an outline of algorithm ACSMF for solving Problem 2 is as follows. http://www.almob.org/content/9/1/9
3. Match the 2k + 4 fragments against the text t using an Aho Corasick automaton [25]. Let L be a list of size Occ of tuples, where < p x , , p t >∈ L is a 3-tuple such that 0 ≤ p x < 2m − 1 is the position where the fragment occurs in x , is the length of the corresponding fragment, and 0 ≤ p t < n is the position where the fragment occurs in t.
4. Compute SA, iSA, LCP, and RMQ LCP of T = x t. Compute SA, iSA, LCP, and RMQ LCP of T r = rev(tx ), that is the reverse string of tx .
5. For each tuple < p x , , p t >∈ L, we try to extend k + 1 times to the right via computing in other words, we compute the length E k r of the longest common prefix of x [p x + . . 2m − 1] and t[p t + . . n − 1], both being suffixes of T, with k mismatches. Similarly, we try to extend to the left k + 1 times via computing E k l using lce queries on the suffixes of T r . 6. For each tuple < p x , , p t >∈ L we try to extend, we also maintain an array M of size 2m − 1, initialised with zeros, where we mark the position of the i -th left and right mismatch, 1 ≤ i ≤ k, by setting

7.
For each E k l , E k r , M computed for tuple < p x , , p t > ∈ L, we report all the valid starting positions in t by first checking if the total length E k l + + E k r ≥ m; that is the length of the full extension of the fragment is greater than or equal to m. If that is the case, then we count the total number of mismatches of the occurrences at starting positions max{p t −E k , p t + −m}, . . . , min{p t + −m+E k r , p t }, by first summing up the mismatches for the leftmost starting position For each subsequent position j + 1, we subtract the value of the leftmost element of M computed for μ j and add the value of the next element to compute μ j+1 . In case μ j ≤ k, we report position j.   Since k < m, we can (pessimistically) replace k by m−1. Then we have 2m(m + 1)n/σ r ≤ cn.
Solving for r, and using k ≤ (2m − 1)/2r − 2, gives the maximum value of k, that is

Algorithm ACSMF-simple
Algorithm ACSMF-Simple is very similar to Algorithm ACSMF. The only differences are: • Algorithm ACSMF-Simple does not perform Step 4 of Algorithm ACSMF; • For each tuple < p x , , p t >∈ L, Step 5 of Algorithm ACSMF is performed without the use of the precomputed indexes. In other words, we compute E k r and E k by simply performing letter comparisons and counting the number of mismatches occurred. The extension stops right before the k + 1th mismatch.

Fact 2. The expected number of letter comparisons required for each extension in algorithm ACSMF-Simple is less than 3.
Proof. Recall that on an alphabet of size σ , the probability that two random strings of length are equal is (1/σ ) . Thus, given two long strings, and setting r = 1/σ , there is probability r that the initial letters are equal, r 2 that the prefixes of length two are equal, and so on. Thus the expected number of positions to be matched before inequality occurs is S = r + 2r 2 + · · · + (n − 1)r n−1 , for some n ≥ 2. Hall & Knight [26, p. 44] tell us that which as n → ∞ approaches r/(1 − r) 2 < 2 for all r. Thus S, the expected number of matching positions, is less than 2, and hence the expected number of letter comparisons required for each extension in algorithm ACSMF-Simple is less than 3. In practical cases, algorithm ACSMF-Simple should be preferred over algorithm ACSMF as (i) it has less memory requirements (see Theorem 3); and (ii) it avoids the construction of a series of data structures (see Section 3 in this regard).

Edit distance model
Algorithm ACSMF-Simple could be easily extended for approximate circular string matching under the edit distance model (for a definition, see [10]). Since each singleletter edit operation can change at most one of the 2k + 4 fragments of x , any set of at most k edit operations leaves at least one of the fragments untouched. In other words, Lemma 2 holds under the edit distance model as well [27]. An area of length O(m) surrounding each potential occurrence found in the filtration phase (Steps 1-3 of algorithm ACSMF) is then searched using the standard dynamicprogramming algorithm in time O(m 2 ) [28] and space O(m) [29]. Since the expected number Occ of occurrences of the 2k + 4 fragments is O( kn

Experimental results
We implemented algorithms ACSMF and ACSMF-Simple as library functions to perform approximate circular string matching with k-mismatches. The functions were implemented in the C programming language and developed under GNU/Linux operating system. They take as input arguments the pattern x of length m, the text t of length n, and the integer threshold k < m; and then return the list of starting positions of the occurrences of the rotations of x in t with k-mismatches as output. The library implementation is distributed under the GNU General http://www.almob.org/content/9/1/9   Public License (GPL), and it is available at http://www. inf.kcl.ac.uk/research/projects/asmf/, which is set up for maintaining the source code and the man-page documentation. The experiments were conducted on a Desktop PC using one core of Intel i7 2600 CPU at 3.4 GHz under GNU/Linux. Approximate circular string matching is a rather undeveloped area. To the best of our knowledge, there does not exist an optimal (average-or worst-case) algorithm for approximate circular string matching with k-mismatches. Therefore, keeping in mind that we wish to evaluate the efficiency of our algorithms in practical terms, we compared their performance to the respective performance of the C implementation a of the optimal average-case algorithm for multiple approximate string matching, presented in [17], for matching the r = m rotations of x. We denote this algorithm by FredNava.
Tables 1, 2, 3 illustrate elapsed-time and speed-up comparisons for various pattern sizes and moderate values of k, using a corpus of DNA data taken from the Pizza & Chili website [30]. As it is demonstrated by the experimental results, algorithm ACSMF-Simple is in all cases the fastest with a speed-up improvement of more than three orders of magnitude over FredNava. ACSMF is always the second fastest, while ACSMF-Simple still retains a speed-up improvement of more than one order of magnitude over ACSMF. Another important observation, also suggested by Corollaries 1 and 2, is that the ACSMF-based algorithms are essentially independent of m for moderate values of k.

Conclusions
In this article, we presented new average-case algorithms for exact and approximate circular string matching. Algorithm ECSMF for exact circular string matching requires average-case time O(n); and Algorithms ACSMF and ACSMF-Simple for approximate circular string matching with k-mismatches require time O(n) for moderate values of k, that is k = O(m/ log σ m). We showed how the same results can be easily obtained under the edit distance model. The presented algorithms were also implemented as library functions. Experimental results demonstrate that the functions provided in this library accelerate the computations by more than three orders of magnitude compared to a naïve approach.
For future work, we will explore the possibility of optimising our algorithms and the corresponding library implementation for the approximate case by using lossless filters for eliminating a possibly large fraction of the input that is guaranteed not to contain any approximate occurrence, such as [31] for the Hamming distance model or [32] for the edit distance model. In addition, we will try to improve our algorithms for the approximate case in order to achieve average-case optimality. Endnote a Personal communication with author.