research-article

Open access

Mesa: a geo-replicated online data warehouse for Google's advertising system

Communications of the ACM, Volume 59, Issue 7

Pages 117 - 125

https://doi.org/10.1145/2936722

Published: 24 June 2016 Publication History

All formats PDF

Abstract

Mesa is a highly scalable analytic data warehousing system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy a complex and challenging set of user and systems requirements, including near real-time data ingestion and retrieval, as well as high availability, reliability, fault tolerance, and scalability for large data and query volumes. Specifically, Mesa handles petabytes of data, processes millions of row updates per second, and serves billions of queries that fetch trillions of rows per day. Mesa is geo-replicated across multiple datacenters and provides consistent and repeatable query answers at low latency, even when an entire datacenter fails. This paper presents the Mesa system and reports the performance and scale that it achieves.

References

[1]

Abouzeid, A., Bajda-Pawlikowski, K., et al. HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads. PVLDB 2, 1 (2009), 922--933.

Digital Library

[2]

Baker, J., Bond, C., et al. Megastore: Providing scalable, highly available storage for interactive services. In CIDR (2011). 223--234.

[3]

Chang, F., Dean, J., et al. Bigtable: A distributed storage system for structured data. In OSDI (2006). 205--218.

Digital Library

[4]

Cohen, J., Eshleman, J., et al. Online expansion of largescale data warehouses. PVLDB 4, 12 (2011), 1249--1259.

[5]

Corbett, J.C., Dean, J., et al. Spanner: Google's globally-distributed database. In OSDI (2012). 251--264.

Digital Library

[6]

Dean, J., Ghemawat, S. MapReduce: Simplified data processing on large clusters. Commun. ACM 51, 1 (2008), 107--113.

Digital Library

[7]

Fikes, A. Storage architecture and challenges. http://goo.gl/pF6kmz, 2010.

[8]

Glendenning, L., Beschastnikh, I., et al. Scalable consistency in scatter. In SOSP (2011). 15--28.

Digital Library

[9]

Gupta, A., Yang, F., et al. Mesa: Geo-replicated, near real-time, scalable data warehousing. In VLDB (2014).

Digital Library

[10]

Lamb, A., Fuller, M., et al. The Vertica analytic database: C-Store 7 years later. PVLDB 5, 12 (2012), 1790--1801.

Digital Library

[11]

Lamport, L. The part-time parliament. ACM Trans. Comput. Syst. 16, 2 (1998), 133--169.

Digital Library

[12]

Lee, G., Lin, J., et al. The unified logging infrastructure for data analytics at Twitter. PVLDB 5, 12 (2012), 1771--1780.

Digital Library

[13]

Project Voldemort: A Distributed Database. http://www.project-voldemort.com/voldemort/.

[14]

SAP HANA. http://www.saphana.com/welcome.

[15]

Shute, J., Vingralek, R., et al. F1: A distributed SQL database that scales. PVLDB 6, 11 (2013), 1068--1079.

Digital Library

[16]

Thusoo, A., Shao, Z., et al. Data warehousing and analytics infrastructure at Facebook. In SIGMOD (2010). 1013--1020.

Digital Library

[17]

Weiss, R. A technical overview of the oracle exadata database machine and exadata storage server. Oracle White Paper. Oracle Corporation, Redwood Shores, 2012.

Cited By

Do TGraefe GNaughton J(2023)Efficient Sorting, Duplicate Removal, Grouping, and AggregationACM Transactions on Database Systems10.1145/356802747:4(1-35)Online publication date: 6-Jan-2023
https://dl.acm.org/doi/10.1145/3568027
Oh KZhang MChandra AWeissman J(2022)Network Cost-Aware Geo-Distributed Data Analytics SystemIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.310889333:6(1407-1420)Online publication date: 1-Jun-2022
https://doi.org/10.1109/TPDS.2021.3108893
Soltani KPadmanabhan AWang S(2022)GeoBalance: workload-aware partitioning of real-time spatiotemporal dataGeoinformatica10.1007/s10707-021-00444-z26:1(67-94)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s10707-021-00444-z
Show More Cited By

Index Terms

Mesa: a geo-replicated online data warehouse for Google's advertising system
1. Information systems
  1. Data management systems
    1. Information integration
      1. Data warehouses
  2. Information systems applications
    1. Data mining

Recommendations

Mesa: geo-replicated, near real-time, scalable data warehousing

Mesa is a highly scalable analytic data warehousing system that stores critical measurement data related to Google's Internet advertising business. Mesa is designed to satisfy a complex and challenging set of user and systems requirements, including ...
Empirical analysis of the mesa instruction set
ASPLOS I: Proceedings of the first international symposium on Architectural support for programming languages and operating systems

This paper describes recent work to refine the instruction set of the Mesa processor. Mesa [8] is a high level systems implementation language developed at Xerox PARC during the middle 1970's. Typical systems written in Mesa are large collections of ...
Empirical analysis of the mesa instruction set

This paper describes recent work to refine the instruction set of the Mesa processor. Mesa [8] is a high level systems implementation language developed at Xerox PARC during the middle 1970's. Typical systems written in Mesa are large collections of ...

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM

Communications of the ACM Volume 59, Issue 7

July 2016

118 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/2963119

Editor:
Moshe Y. Vardi
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Copyright © 2016 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 June 2016

Published in CACM Volume 59, Issue 7

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
23,350
Total Downloads

Downloads (Last 12 months)573
Downloads (Last 6 weeks)61

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Do TGraefe GNaughton J(2023)Efficient Sorting, Duplicate Removal, Grouping, and AggregationACM Transactions on Database Systems10.1145/356802747:4(1-35)Online publication date: 6-Jan-2023
https://dl.acm.org/doi/10.1145/3568027
Oh KZhang MChandra AWeissman J(2022)Network Cost-Aware Geo-Distributed Data Analytics SystemIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.310889333:6(1407-1420)Online publication date: 1-Jun-2022
https://doi.org/10.1109/TPDS.2021.3108893
Soltani KPadmanabhan AWang S(2022)GeoBalance: workload-aware partitioning of real-time spatiotemporal dataGeoinformatica10.1007/s10707-021-00444-z26:1(67-94)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s10707-021-00444-z
Agiwal ALai KManoharan GRoy ISankaranarayanan JZhang HZou TChen MChen ZDai MDo TGao HGeng HGrover RHuang BHuang YLi ZLiang JLin TLiu LLiu YMao XMeng YMishra PPatel JR. RRaman VRoy SShishodia MSun TTang YTatemura JTrehan SVadali RVenkatasubramanian PZhang GZhang KZhang YZhuang ZGraefe GAgrawal DNaughton JKosalge SHacıgümüş H(2021)NapaProceedings of the VLDB Endowment10.14778/3476311.347637714:12(2986-2997)Online publication date: 1-Jul-2021
https://dl.acm.org/doi/10.14778/3476311.3476377
Oh KChandra AWeissman J(2020)A Network Cost-aware Geo-distributed Data Analytics System2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID)10.1109/CCGrid49817.2020.00-28(649-658)Online publication date: May-2020
https://doi.org/10.1109/CCGrid49817.2020.00-28
Mehmood EAnees T(2020)Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2020.30052688(119123-119143)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3005268
Ngo VLe-Khac NKechadi M(2019)Designing and Implementing Data Warehouse for Agricultural Big DataBig Data – BigData 201910.1007/978-3-030-23551-2_1(1-17)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1007/978-3-030-23551-2_1
Ghosh MRaina AXu LQian XGupta IGupta HOliveira RFelber PHu Y(2018)Popular is cheaperProceedings of the Thirteenth EuroSys Conference10.1145/3190508.3190542(1-14)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3190508.3190542
Noormohammadpour MRaghavendra CRao SKandula SDe Lara ESundararaman S(2017)DCcastProceedings of the 9th USENIX Conference on Hot Topics in Cloud Computing10.5555/3154580.3154595(15-15)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.5555/3154580.3154595
Mortazavi SSalehe MGomes CPhillips Cde Lara EZhang JChiang MMaggs B(2017)CloudpathProceedings of the Second ACM/IEEE Symposium on Edge Computing10.1145/3132211.3134464(1-13)Online publication date: 12-Oct-2017
https://dl.acm.org/doi/10.1145/3132211.3134464

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents