State-of-the-art database systems manage and process a variety of complex objects, including strings and trees. For such objects equality comparisons are often not meaningful and must be replaced by similarity comparisons. This book describes the concepts and techniques to incorporate similarity into database systems. We start out by discussing the properties of strings and trees, and identify the edit distance as the de facto standard for comparing complex objects. Since the edit distance is computationally expensive, token-based distances have been introduced to speed up edit distance computations. The basic idea is to decompose complex objects into sets of tokens that can be compared efficiently. Token-based distances are used to compute an approximation of the edit distance and prune expensive edit distance calculations. A key observation when computing similarity joins is that many of the object pairs, for which the similarity is computed, are very different from each other. Filters exploit this property to improve the performance of similarity joins. A filter preprocesses the input data sets and produces a set of candidate pairs. The distance function is evaluated on the candidate pairs only. We describe the essential query processing techniques for filters based on lower and upper bounds. For token equality joins we describe prefix, size, positional and partitioning filters, which can be used to avoid the computation of small intersections that are not needed since the similarity would be too low. Table of Contents: Preface / Acknowledgments / Introduction / Data Types / Edit-Based Distances / Token-Based Distances / Query Processing Techniques / Filters for Token Equality Joins / Conclusion / Bibliography / Authors' Biographies / Index
Cited By
- Wang J, Yang J and Zhang W Top-k Tree Similarity Join Proceedings of the 30th ACM International Conference on Information & Knowledge Management, (1939-1948)
- Guan S, Ma H and Wu Y Attribute-Driven Backbone Discovery Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, (187-195)
- Perdacher M, Plant C and Böhm C Cache-oblivious High-performance Similarity Join Proceedings of the 2019 International Conference on Management of Data, (87-104)
- McCauley S and Silvestri F Adaptive MapReduce Similarity Joins Proceedings of the 5th ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond, (1-4)
- McCauley S, Mikkelsen J and Pagh R Set Similarity Search for Skewed Data Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, (63-74)
- Aumüller M, Christiani T, Pagh R and Silvestri F Distance-Sensitive Hashing Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, (89-104)
- Chang L, Li W, Qin L, Zhang W and Yang S (2017). $\mathsf {pSCAN}$, IEEE Transactions on Knowledge and Data Engineering, 29:2, (387-401), Online publication date: 1-Feb-2017.
- Ahle T, Pagh R, Razenshteyn I and Silvestri F On the Complexity of Inner Product Similarity Join Proceedings of the 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, (151-164)
- Mann W, Augsten N and Bouros P (2016). An empirical evaluation of set similarity join techniques, Proceedings of the VLDB Endowment, 9:9, (636-647), Online publication date: 1-May-2016.
- Tang Y, Cai Y and Mamoulis N (2015). Scaling similarity joins over tree-structured data, Proceedings of the VLDB Endowment, 8:11, (1130-1141), Online publication date: 1-Jul-2015.
- Tang Y, Cai Y and Mamoulis N (2018). Scaling similarity joins over tree-structured data, Proceedings of the VLDB Endowment, 8:11, (1130-1141), Online publication date: 1-Jul-2015.
Recommendations
On-the-fly token similarity joins in relational databases
SIGMOD '14: Proceedings of the 2014 ACM SIGMOD International Conference on Management of DataToken similarity joins represent data items as sets of tokens, for example, strings are represented as sets of q-grams (substrings of length q). Two items are considered similar and match if their token sets have a large overlap. Previous work on ...
Similarity joins as stronger metric operations
Similarity joins between two sets of records return pairs of records whose similarity is no less than a given threshold. More specifically, consider two sets of records, R and S, a similarity function sim(.,.) and a threshold t, a similarity join ...
Similarity Joins
Similarity Joins are extensively used in multiple application domains and are recognized among the most useful data processing and analysis operations. They retrieve all data pairs whose distances are smaller than a predefined threshold ε. While several ...