Article

Sorting, searching, and simulation in the mapreduce framework

Authors:

Michael T. Goodrich,

Nodari Sitchinava,

Qin ZhangAuthors Info & Claims

ISAAC'11: Proceedings of the 22nd international conference on Algorithms and Computation

Pages 374 - 383

https://doi.org/10.1007/978-3-642-25591-5_39

Published: 05 December 2011 Publication History

Publisher Site

Abstract

We study the MapReduce framework from an algorithmic standpoint, providing a generalization of the previous algorithmic models for MapReduce. We present optimal solutions for the fundamental problems of all-prefix-sums, sorting and multi-searching. Additionally, we design optimal simulations of the the well-established PRAM and BSP models in MapReduce, immediately resulting in optimal solutions to the problems of computing fixed-dimensional linear programming and 2-D and 3-D convex hulls.

References

[1]

Alon, N., Megiddo, N.: Parallel linear programming in fixed dimension almost surely in constant time. J. ACM 41(2), 422-434 (1994).

Crossref

Google Scholar

[2]

Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107-113 (2008).

Crossref

Google Scholar

[3]

Dean, J., Ghemawat, S.: MapReduce: a flexible data processing tool. Commun. ACM 53(1), 72-77 (2010).

Crossref

Google Scholar

[4]

DeWitt, D.J., Stonebraker, M.: MapReduce: A major step backwards. Database Column (2008), http://databasecolumn.vertica.com/database-innovation/ mapreduce-a-major-step-backwards/.

Google Scholar

[5]

Eisenstat, S.C.: O (log* n ) algorithms on a Sum-CRCW PRAM. Computing 79(1), 93-97 (2007).

Crossref

Google Scholar

[6]

Feldman, J., Muthukrishnan, S., Sidiropoulos, A., Stein, C., Svitkina, Z.: On distributing symmetric streaming computations. In: Teng, S.H. (ed.) SODA, pp. 710-719. SIAM (2008).

Crossref

Google Scholar

[7]

Goodrich, M.T.: Randomized fully-scalable BSP techniques for multi-searching and convex hull construction. In: SODA, pp. 767-776 (1997).

Crossref

Google Scholar

[8]

Goodrich, M.T.: Communication-efficient parallel sorting. SIAM Journal on Computing 29(2), 416-432 (1999).

Crossref

Google Scholar

[9]

Goodrich, M.T., Sitchinava, N., Zhang, Q.: Sorting, searching, and simulation in the mapreduce framework (2011), http://arxiv.org/abs/1101.1902.

Google Scholar

[10]

JáJá, J.: An Introduction to Parallel Algorithms. Addison-Wesley, Reading (1992).

Crossref

Google Scholar

[11]

Kannan, S., Naor, M., Rudich, S.: Implicit representation of graphs. In: 20th Annual ACM Symposium on Theory of Computing (STOC), pp. 334-343 (1988).

Crossref

Google Scholar

[12]

Karloff, H., Suri, S., Vassilvitskii, S.: A model of computation for MapReduce. In: Proc. ACM-SIAM Sympos. Discrete Algorithms (SODA), pp. 938-948 (2010).

Crossref

Google Scholar

[13]

Valiant, L.G.: A bridging model for parallel computation. Comm. ACM 33, 103-111 (1990).

Crossref

Google Scholar

Cited By

View all

Ji HPai SPemmaraju SSobel J(2025)Faster Set Cover in the MPC ModelProceedings of the 26th International Conference on Distributed Computing and Networking10.1145/3700838.3700861(144-151)Online publication date: 4-Jan-2025
https://dl.acm.org/doi/10.1145/3700838.3700861
Sanford CHsu DTelgarsky MSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Transformers, parallel computation, and logarithmic depthProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693833(43276-43327)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693833
Mitrović SPan TSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Faster streaming and scalable algorithms for finding directed dense subgraphs in large graphsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693532(35876-35891)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693532
Show More Cited By

Sorting, searching, and simulation in the mapreduce framework
1. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
2. Theory of computation
  1. Models of computation
    1. Concurrency

Recommendations

Implementation of Distributed Searching and Sorting using Hadoop MapReduce
ICTCS '14: Proceedings of the 2014 International Conference on Information and Communication Technology for Competitive Strategies

This paper focuses on implementation of MapReduce programming model on Hadoop cluster for parallel processing of huge amount of data efficiently. There is deluge of data everywhere and we need to process these data efficiently to take decisions and to ...
MapReduce: Review and open challenges

The continuous increase in computational capacity over the past years has produced an overwhelming flow of data or big data, which exceeds the capabilities of conventional processing tools. Big data signify a new era in data exploration and utilization. ...
Big Data Analysis Solutions Using MapReduce Framework
ICCCE '14: Proceedings of the 2014 International Conference on Computer and Communication Engineering

Recently, data that generated from variety of sources with massive volumes, high rates, and different data structure, data with these characteristics is called Big Data. Big Data processing and analyzing is a challenge for the current systems because ...

Comments

Information & Contributors

Information

Published In

ISAAC'11: Proceedings of the 22nd international conference on Algorithms and Computation

December 2011

775 pages

ISBN:9783642255908

Editors:
Takao Asano
Chuo University, Kasuga, Bunkyo-ku, Tokyo, Japan
,
Shin-ichi Nakano
Gunma University, 1-5-1 Tenjin-Cho, Kiryu-Shi, Japan
,
Yoshio Okamoto
Japan Advanced Institute of Science and Technology, 1-1 Asahidai, Nomi, Ishikawa, Japan
,
Osamu Watanabe
Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo, Ishikawa, Japan

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 December 2011

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

87
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ji HPai SPemmaraju SSobel J(2025)Faster Set Cover in the MPC ModelProceedings of the 26th International Conference on Distributed Computing and Networking10.1145/3700838.3700861(144-151)Online publication date: 4-Jan-2025
https://dl.acm.org/doi/10.1145/3700838.3700861
Sanford CHsu DTelgarsky MSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Transformers, parallel computation, and logarithmic depthProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693833(43276-43327)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693833
Mitrović SPan TSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Faster streaming and scalable algorithms for finding directed dense subgraphs in large graphsProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693532(35876-35891)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693532
La Tour MHenzinger MSaulpic DSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Making old things newProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692550(12046-12086)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692550
Dalirrooyfard MMakarychev KMitrović SSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Pruned PivotProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692464(9931-9952)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692464
Luo SZhu ZLarson K(2024)Massively parallel single-source SimRanks in o(log n) roundsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/249(2252-2260)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/249
Liu QSeshadhri CKuznetsov PGelles ROlivetti D(2024)Brief Announcement: Improved Massively Parallel Triangle Counting in O(1) RoundsProceedings of the 43rd ACM Symposium on Principles of Distributed Computing10.1145/3662158.3662819(519-522)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3662158.3662819
Giliberti JParsaeian ZKuznetsov PGelles ROlivetti D(2024)Brief Announcement: Massively Parallel Ruling Set Made DeterministicProceedings of the 43rd ACM Symposium on Principles of Distributed Computing10.1145/3662158.3662816(523-526)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3662158.3662816
Czumaj AMishra GMukherjee AKuznetsov PGelles ROlivetti D(2024)Streaming Graph Algorithms in the Massively Parallel Computation ModelProceedings of the 43rd ACM Symposium on Principles of Distributed Computing10.1145/3662158.3662770(496-507)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3662158.3662770
Koo JAgrawal KPetrank E(2024)An Optimal MPC Algorithm for Subunit-Monge Matrix Multiplication, with Applications to LISProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659974(145-154)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659974
Show More Cited By

Abstract

References

Cited By

Recommendations

Implementation of Distributed Searching and Sorting using Hadoop MapReduce

MapReduce: Review and open challenges

Big Data Analysis Solutions Using MapReduce Framework

Comments

Information

Published In

Sponsors

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations