Article

Asymmetric LSH (ALSH) for sublinear time Maximum Inner Product Search (MIPS)

Authors:

Anshumali Shrivastava,

Ping LiAuthors Info & Claims

NIPS'14: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2

Pages 2321 - 2329

Published: 08 December 2014 Publication History

Abstract

We present the first provably sublinear time hashing algorithm for approximate Maximum Inner Product Search (MIPS). Searching with (un-normalized) inner product as the underlying similarity measure is a known difficult problem and finding hashing schemes for MIPS was considered hard. While the existing Locality Sensitive Hashing (LSH) framework is insufficient for solving MIPS, in this paper we extend the LSH framework to allow asymmetric hashing schemes. Our proposal is based on a key observation that the problem of finding maximum inner products, after independent asymmetric transformations, can be converted into the problem of approximate near neighbor search in classical settings. This key observation makes efficient sublinear hashing scheme for MIPS possible. Under the extended asymmetric LSH (ALSH) framework, this paper provides an example of explicit construction of provably fast hashing scheme for MIPS. Our proposed algorithm is simple and easy to implement. The proposed hashing scheme leads to significant computational savings over the two popular conventional LSH schemes: (i) Sign Random Projection (SRP) and (ii) hashing based on p-stable distributions for L₂ norm (L2LSH), in the collaborative filtering task of item recommendations on Netflix and Movielens (10M) datasets.

References

[1]

A. Andoni and P. Indyk. E2lsh: Exact euclidean locality sensitive hashing. Technical report, 2004.

[2]

A. Z. Broder. On the resemblance and containment of documents. In the Compression and Complexity of Sequences, pages 21-29, Positano, Italy, 1997.

Digital Library

[3]

M. S. Charikar. Similarity estimation techniques from rounding algorithms. In STOC, pages 380-388, Montreal, Quebec, Canada, 2002.

Digital Library

[4]

P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-n recommendation tasks. In Proceedings of the fourth ACM conference on Recommender systems, pages 39-46. ACM, 2010.

Digital Library

[5]

R. R. Curtin, A. G. Gray, and P. Ram. Fast exact max-kernel search. In SDM, pages 1-9, 2013.

[6]

M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokn. Locality-sensitive hashing scheme based on p-stable distributions. In SCG, pages 253-262, Brooklyn, NY, 2004.

Digital Library

[7]

T. Dean, M. A. Ruzon, M. Segal, J. Shlens, S. Vijayanarasimhan, and J. Yagnik. Fast, accurate detection of 100,000 object classes on a single machine. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 1814-1821. IEEE, 2013.

Digital Library

[8]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(9):1627-1645, 2010.

Digital Library

[9]

J. H. Friedman and J. W. Tukey. A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on Computers, 23(9):881-890, 1974.

Digital Library

[10]

M. X. Goemans and D. P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of ACM, 42(6):1115-1145, 1995.

Digital Library

[11]

S. Har-Peled, P. Indyk, and R. Motwani. Approximate nearest neighbor: Towards removing the curse of dimensionality. Theory of Computing, 8(14):321-350, 2012.

[12]

P. Indyk and R. Motwani. Approximate nearest neighbors: Towards removing the curse of dimensionality. In STOC, pages 604-613, Dallas, TX, 1998.

Digital Library

[13]

N. Koenigstein, P. Ram, and Y. Shavitt. Efficient retrieval of recommendations in a matrix factorization framework. In CIKM, pages 535-544, 2012.

Digital Library

[14]

Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems.

[15]

P. Li and A. C. König. Theory and applications b-bit minwise hashing. Commun. ACM, 2011.

Digital Library

[16]

P. Li, M. Mitzenmacher, and A. Shrivastava. Coding for random projections. In ICML, 2014.

[17]

P. Li, M. Mitzenmacher, and A. Shrivastava. Coding for random projections and approximate near neighbor search. Technical report, arXiv:1403.8144, 2014.

[18]

B. Neyshabur and N. Srebro. A simpler and better lsh for maximum inner product search (mips). Technical report, arXiv:1410.5518, 2014.

[19]

P. Ram and A. G. Gray. Maximum inner-product search using cone trees. In KDD, pages 931-939, 2012.

Digital Library

[20]

A. Shrivastava and P. Li. Beyond pairwise: Provably fast algorithms for approximate k-way similarity search. In NIPS, Lake Tahoe, NV, 2013.

Digital Library

[21]

A. Shrivastava and P. Li. Asymmetric minwise hashing. Technical report, 2014.

[22]

A. Shrivastava and P. Li. An improved scheme for asymmetric lsh. Technical report, arXiv:1410.5410, 2014.

[23]

A. Shrivastava and P. Li. In defense of minhash over simhash. In AISTATS, 2014.

[24]

R. Weber, H.-J. Schek, and S. Blott. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In VLDB, pages 194-205, 1998.

Digital Library

Cited By

Lee JDai ZDuddu SLei TNaim IChang MZhao VOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Rethinking the role of token retrieval in multi-vector retrievalProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666799(15384-15405)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666799
Le DLauw H(2021)Efficient Retrieval of Matrix Factorization-Based Top-k RecommendationsJournal of Artificial Intelligence Research10.1613/jair.1.1240370(1441-1479)Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1613/jair.1.12403
Gao WFan XWang CSun JJia KXiao WDing RBin XYang HLiu XDemartini GZuccon GCulpepper JHuang ZTong H(2021)Learning An End-to-End Structure for Retrieval in Large-Scale RecommendationsProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482362(524-533)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482362
Show More Cited By

Index Terms

Asymmetric LSH (ALSH) for sublinear time Maximum Inner Product Search (MIPS)
1. Information systems
  1. Information retrieval
  2. Information systems applications
    1. Data mining
      1. Nearest-neighbor search
2. Theory of computation
  1. Design and analysis of algorithms
    1. Data structures design and analysis
      1. Sorting and searching

Index terms have been assigned to the content through auto-classification.

Recommendations

Improved asymmetric locality sensitive hashing (ALSH) for Maximum Inner Product Search (MIPS)
UAI'15: Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence

Recently we showed that the problem of Maximum Inner Product Search (MIPS) is efficient and it admits provably sub-linear hashing algorithms. In [23], we used asymmetric transformations to convert the problem of approximate MIPS into the problem of ...
Accurate and Fast Asymmetric Locality-Sensitive Hashing Scheme for Maximum Inner Product Search
KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

The problem of Approximate Maximum Inner Product (AMIP) search has received increasing attention due to its wide applications. Interestingly, based on asymmetric transformation, the problem can be reduced to the Approximate Nearest Neighbor (ANN) search,...
Multi-probe LSH: efficient indexing for high-dimensional similarity search
VLDB '07: Proceedings of the 33rd international conference on Very large data bases

Similarity indices for high-dimensional data are very desirable for building content-based search systems for feature-rich data such as audio, images, videos, and other sensor data. Recently, locality sensitive hashing (LSH) and its variations have been ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'14: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2

December 2014

3697 pages

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 08 December 2014

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
2
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee JDai ZDuddu SLei TNaim IChang MZhao VOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Rethinking the role of token retrieval in multi-vector retrievalProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666799(15384-15405)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666799
Le DLauw H(2021)Efficient Retrieval of Matrix Factorization-Based Top-k RecommendationsJournal of Artificial Intelligence Research10.1613/jair.1.1240370(1441-1479)Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1613/jair.1.12403
Gao WFan XWang CSun JJia KXiao WDing RBin XYang HLiu XDemartini GZuccon GCulpepper JHuang ZTong H(2021)Learning An End-to-End Structure for Retrieval in Large-Scale RecommendationsProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482362(524-533)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482362
Ma CYu FYu YLi WDemartini GZuccon GCulpepper JHuang ZTong H(2021)Learning Sparse Binary Code for Maximum Inner Product SearchProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482132(3308-3312)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482132
Feng WDeng DLi GLi ZIdreos SSrivastava D(2021)Allign: Aligning All-Pair Near-Duplicate Passages in Long TextsProceedings of the 2021 International Conference on Management of Data10.1145/3448016.3457548(541-553)Online publication date: 9-Jun-2021
https://dl.acm.org/doi/10.1145/3448016.3457548
Pham NZhu FChin Ooi BMiao CWang HSkrypnyk IHsu WChawla S(2021)Simple Yet Efficient Algorithms for Maximum Inner Product Search via Extreme Order StatisticsProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3467345(1339-1347)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3467345
Gao LDai ZChen TFan ZVan Durme BCallan J(2021)Complement Lexical Retrieval Model with Semantic Residual EmbeddingsAdvances in Information Retrieval10.1007/978-3-030-72113-8_10(146-160)Online publication date: 28-Mar-2021
https://dl.acm.org/doi/10.1007/978-3-030-72113-8_10
Guu KLee KTung ZPasupat PChang MDaumé HSingh A(2020)REALMProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525306(3929-3938)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525306
Guo RSun PLindgren EGeng QSimcha DChern FKumar SDaumé HSingh A(2020)Accelerating large-scale inference with anisotropic vector quantizationProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525302(3887-3896)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525302
Wang RDeng D(2020)DeltaPQProceedings of the VLDB Endowment10.14778/3424573.342458013:13(3603-3616)Online publication date: 27-Oct-2020
https://dl.acm.org/doi/10.14778/3424573.3424580
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents