Article

On Optimizing Nearest Neighbor Queries in High-Dimensional Data Spaces

Authors:

Stefan Berchtold,

Christian Böhm,

Daniel A. Keim,

Hans-Peter KriegelAuthors Info & Claims

ICDT '01: Proceedings of the 8th International Conference on Database Theory

Pages 435 - 449

Published: 04 January 2001 Publication History

Abstract

Nearest-neighbor queries in high-dimensional space are of high importance in various applications, especially in content-based indexing of multimedia data. For an optimization of the query processing, accurate models for estimating the query processing costs are needed. In this paper, we propose a new cost model for nearest neighbor queries in high-dimensional space, which we apply to enhance the performance of high-dimensional index structures. The model is based on new insights into effects occurring in high-dimensional space and provides a closed formula for the processing costs of nearest neighbor queries depending on the dimensionality, the block size and the database size. From the wide range of possible applications of our model, we select two interesting samples: First, we use the model to prove the known linear complexity of the nearest neighbor search problem in high-dimensional space, and second, we provide a technique for optimizing the block size. For data of medium dimensionality, the optimized block size allows significant speed-ups of the query processing time when compared to traditional block sizes and to the linear scan.

References

[1]

Altschul S. F., Gish W., Miller W., Myers E. W., Lipman D. J.: 'A Basic Local Alignment Search Tool', Journal of Molecular Biology, Vol. 215, No. 3, 1990, pp. 403- 410.

[2]

Beckmann N., Kriegel H.-P., Schneider R., Seeger B.: 'The R*-tree: An Efficient and Robust Access Method for Points and Rectangles', Proc. ACM SIGMOD Int. Conf. on Management of Data, Atlantic City, NJ, 1990, pp. 322-331.

[3]

Berchtold S., Böhm C., Keim D., Kriegel H.-P.: 'A Cost Model For Nearest Neighbor Search in High-Dimensional Data Space', Proc. ACM PODS Int. Conf. on Principles of Databases, Tucson, Arizona, 1997.

[4]

Berchtold S., Böhm C., Braunmüller B., Keim D., Kriegel H.-P.: 'Fast Parallel Similarity Search in Multimedia Databases', Proc. ACM SIGMOD Int. Conf. on Management of Data, Tucson, Arizona, 1997.

[5]

Berchtold S., Keim D. A.: 'High-dimensional Index Structures: Database Support for Next Decades's Applications', Tutorial, Proc. ACM SIGMOD Int. Conf. on Management of Data, 1998, p. 501.

[6]

Berchtold S., Keim D., Kriegel H.-P.: 'The X-tree: An Index Structure for High-Dimensional Data', 22nd Conf. on Very Large Databases, 1996, Bombay, India.

[7]

Berchtold S., Keim D., Kriegel H.-P.: 'Fast Searching for Partial Similarity in Polygon Databases', VLDB Journal, Dec. 1997.

[8]

Ciacia P., Patella M., Zezula P.: 'A Cost Model for Similarity Queries in Metric Spaces', Proc. ACM PODS Int. Conf. on Principals of Databases, Seattle, WA, 1998, pp. 59-68.

[9]

Cleary J. G.: 'Analysis of an Algorithm for Finding Nearest Neighbors in Euclidean Space', ACM Transactions on Mathematical Software, Vol. 5, No. 2, June 1979, pp.183-192.

[10]

Faloutsos C., Barber R., Flickner M., Hafner J., et al.: 'Efficient and Effective Querying by Image Content', Journal of Intelligent Information Systems, 1994, Vol. 3, pp. 231-262.

[11]

Friedman J. H., Bentley J. L., Finkel R. A.: 'An Algorithm for Finding Best Matches in Logarithmic Expected Time', ACM Transactions on Mathematical Software, Vol. 3, No. 3, September 1977, pp. 209-226.

[12]

Hjaltason G. R., Samet H.: 'Ranking in Spatial Databases', Proc. 4th Int. Symp. on Large Spatial Databases, Portland, ME, 1995, pp. 83-95.

[13]

Katayama N., Satoh S.: 'The SR-Tree: An Index Structure for High-Dimensional Nearest Neighbor Queries', Proc. ACM SIGMOD Int. Conf. on Management of Data, 1997.

[14]

Kukich K.: 'Techniques for Automatically Correcting Words in Text', ACM Computing Surveys, Vol. 24, No. 4, 1992, pp. 377-440.

[15]

Jagadish H. V.: 'A Retrieval Technique for Similar Shapes', Proc. ACM SIGMOD Int. Conf. on Management of Data, 1991, pp. 208-217.

[16]

Lin K., Jagadish H. V., Faloutsos C.: 'The TV-tree: An Index Structure for High-Dimensional Data', VLDB Journal, Vol. 3, 1995, pp. 517-542.

[17]

Mehrotra R., Gary J. E.: 'Feature-Based Retrieval of Similar Shapes', Proc. 9th Int. Conf. on Data Engineering, Vienna, Austria, 1993, pp. 108-115.

[18]

Mehrotra R., Gary J. E.: 'Feature-Index-Based Similar Shape Retrieval', Proc. of the 3rd Working Conf. on Visual Database Systems, March 1995.

[19]

Roussopoulos N., Kelley S., Vincent F.: 'Nearest Neighbor Queries', Proc. ACM SIGMOD Int. Conf. on Management of Data, 1995, pp. 71-79.

[20]

Shawney H., Hafner J.: 'Efficient Color Histogram Indexing', Proc. Int. Conf. on Image Processing, 1994, pp. 66-70.

[21]

Shoichet B. K., Bodian D. L., Kuntz I. D.: 'Molecular Docking Using Shape Descriptors', Journal of Computational Chemistry, Vol. 13, No. 3, 1992, pp. 380-397.

[22]

Sproull R.F.: 'Refinements to Nearest Neighbor Searching in k-Dimensional Trees', Algorithmica 1991, pp. 579-589.

[23]

Wallace T., Wintz P.: 'An Efficient Three-Dimensional Aircraft Recognition Algorithm Using Normalized Fourier Descriptors', Computer Graphics and Image Processing, Vol. 13, pp. 99-126, 1980.

[24]

Weber R., Schek H.-J., Blott S.: 'A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces', Proc. Int. Conf. on Very Large Databases, New York, 1998.

[25]

White, D., Jain R.: 'Similarity Indexing with the SS-Tree', Proc. 12th Int. Conf. on Data Engineering, New Orleans, LA, 1996, pp. 516-523.

Cited By

Kaster DBugatti PPonciano-Silva MTraina AMarques PSantos ATraina C(2011)MedFMI-SiRProceedings of the Second international conference on Information technology in bio- and medical informatics10.5555/2035485.2035488(16-30)Online publication date: 31-Aug-2011
https://dl.acm.org/doi/10.5555/2035485.2035488
Adler MHeeringa B(2008)Search space reductions for nearest-neighbor queriesProceedings of the 5th international conference on Theory and applications of models of computation10.5555/1791834.1791885(554-567)Online publication date: 25-Apr-2008
https://dl.acm.org/doi/10.5555/1791834.1791885
Zhuang YZhuang YLi QChen L(2007)Interactive high-dimensional index for large Chinese calligraphic character databasesACM Transactions on Asian Language Information Processing10.1145/1282080.12820836:2(8-es)Online publication date: 1-Sep-2007
https://dl.acm.org/doi/10.1145/1282080.1282083
Show More Cited By

On Optimizing Nearest Neighbor Queries in High-Dimensional Data Spaces
1. Information systems
  1. Data management systems
    1. Database management system engines
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory

Recommendations

High dimensional reverse nearest neighbor queries
CIKM '03: Proceedings of the twelfth international conference on Information and knowledge management

Reverse Nearest Neighbor (RNN) queries are of particular interest in a wide range of applications such as decision support systems, profile based marketing, data streaming, document databases, and bioinformatics. The earlier approaches to solve this ...
An Algorithm for Incremental Nearest Neighbor Search in High-Dimensional Data Spaces
Proceedings of the First International Conference on The Human Society and the Internet - Internet Related Socio-Economic Issues

The SPY-TEC (Spherical Pyramid-Technique) [8] was proposed as a new indexing method for high-dimensional data spaces using a special partitioning strategy that divides a d-dimensional data space into 2d spherical pyramids. Although the authors of [8] ...
Grassmann Hashing for approximate nearest neighbor search in high dimensional space
ICME '11: Proceedings of the 2011 IEEE International Conference on Multimedia and Expo

Locality-Sensitive Hashing (LSH) approximates nearest neighbors in high dimensions by projecting original data into low-dimensional subspaces. The basic idea is to hash data samples to ensure that the probability of collision is much higher for samples ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICDT '01: Proceedings of the 8th International Conference on Database Theory

January 2001

449 pages

ISBN:3540414568

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 04 January 2001

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kaster DBugatti PPonciano-Silva MTraina AMarques PSantos ATraina C(2011)MedFMI-SiRProceedings of the Second international conference on Information technology in bio- and medical informatics10.5555/2035485.2035488(16-30)Online publication date: 31-Aug-2011
https://dl.acm.org/doi/10.5555/2035485.2035488
Adler MHeeringa B(2008)Search space reductions for nearest-neighbor queriesProceedings of the 5th international conference on Theory and applications of models of computation10.5555/1791834.1791885(554-567)Online publication date: 25-Apr-2008
https://dl.acm.org/doi/10.5555/1791834.1791885
Zhuang YZhuang YLi QChen L(2007)Interactive high-dimensional index for large Chinese calligraphic character databasesACM Transactions on Asian Language Information Processing10.1145/1282080.12820836:2(8-es)Online publication date: 1-Sep-2007
https://dl.acm.org/doi/10.1145/1282080.1282083
Cui BShen HShen JTan K(2005)Exploring bit-difference for approximate KNN search in high-dimensional databasesProceedings of the 16th Australasian database conference - Volume 3910.5555/1082222.1082240(165-174)Online publication date: 30-Jan-2005
https://dl.acm.org/doi/10.5555/1082222.1082240
Cui BOoi BSu JTan K(2005)Indexing High-Dimensional Data for Efficient In-Memory Similarity SearchIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2005.4617:3(339-353)Online publication date: 1-Mar-2005
https://dl.acm.org/doi/10.1109/TKDE.2005.46
Tao YZhang JPapadias DMamoulis N(2004)An Efficient Cost Model for Optimization of Nearest Neighbor Search in Low and Medium Dimensional SpacesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2004.4816:10(1169-1184)Online publication date: 1-Oct-2004
https://dl.acm.org/doi/10.1109/TKDE.2004.48
Tao YSun JPapadias D(2003)Analysis of predictive spatio-temporal queriesACM Transactions on Database Systems10.1145/958942.95894328:4(295-336)Online publication date: 1-Dec-2003
https://dl.acm.org/doi/10.1145/958942.958943
Zhang JZhu MPapadias DTao YLee DIves ZPapakonstantinou YHalevy A(2003)Location-based spatial queriesProceedings of the 2003 ACM SIGMOD international conference on Management of data10.1145/872757.872812(443-454)Online publication date: 9-Jun-2003
https://dl.acm.org/doi/10.1145/872757.872812
Tao YPapadias D(2003)Spatial queries in dynamic environmentsACM Transactions on Database Systems10.1145/777943.77794428:2(101-139)Online publication date: 1-Jun-2003
https://dl.acm.org/doi/10.1145/777943.777944
Tao YPapadias DShen Q(2002)Continuous nearest neighbor searchProceedings of the 28th international conference on Very Large Data Bases10.5555/1287369.1287395(287-298)Online publication date: 20-Aug-2002
https://dl.acm.org/doi/10.5555/1287369.1287395
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten