StarFlow: fine-grained execution of workflows in Hybrid Cloud HPC for data stream applications
Article No.: 64, Pages 1 - 2
Abstract
Nowadays we are witnessing the diffusion of the Function-as-a-service paradigm, that has emerged due to its ability to deploy fine-grained units of computation on heterogeneous resources on-demand, reducing costs and improving resource utilization. In this work, we discuss our objective: extending the runtime support of a structured parallel programming framework, FastFlow, to empower it with the mechanisms needed to exploit FaaS functionalities. To achieve this objective, we also integrates workflows as a way to specify and describe, at a very high level of abstraction, the topology and the requirements of complex parallel data stream applications.
References
[1]
Sonia Campa, Marco Danelutto, Mehdi Goli, Horacio González-Vélez, Alina Madalina Popescu, and Massimo Torquati. 2014. Parallel patterns for heterogeneous CPU/GPU architectures: Structured parallelism from cluster to cloud. Future Generation Computer Systems 37 (2014), 354--366.
[2]
Iacopo Colonnelli, B Cantalupo, I Merelli, and Marco Aldinucci. 2021. StreamFlow: cross-breeding cloud with HPC. IEEE Transactions on Emerging Topics in Computing 9 (2021), 1723--1737.
[3]
M.R. Crusoe, S. Abeln, A. Iosup, P. Amstutz, J. Chilton, N. Tijanic, H. Ménager, S. Soiland-Reyes, B. Gavrilovic, Goblem C.A., and the CWL Community. 2022. Methods included: standardizing computational reuse and portability with the Common Workflow Language. Commun. ACM 65 (2022), 54--63.
[4]
Marco Danelutto and Massimo Torquati. 2014. Loop Parallelism: A New Skeleton Perspective on Data Parallel Patterns. In 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing. 52--59.
[5]
Gabriele Russo Russo, Tiziana Mannucci, Valeria Cardellini, and Francesco Lo Presti. 2023. Serverledge: Decentralized Function-as-a-Service for the Edge-Cloud Continuum. In 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom). 131--140.
[6]
Johann Schleier-Smith, Vikram Sreekanti, Anurag Khandelwal, Joao Carreira, Neeraja J Yadwadkar, Raluca Ada Popa, Joseph E Gonzalez, Ion Stoica, and David A Patterson. 2021. What serverless computing is and should become: The next phase of cloud computing. Commun. ACM 64, 5 (2021), 76--84.
Index Terms
- StarFlow: fine-grained execution of workflows in Hybrid Cloud HPC for data stream applications
Recommendations
Challenges of Running Scientific Workflows in Cloud Environments
ScienceCloud '15: Proceedings of the 6th Workshop on Scientific Cloud ComputingThis talk will touch upon the challenges of running scientific workflows in distributed environments such as academic and commercial clouds. It will describe the Pegasus Workflow Management System [1] and how it manages the execution of a variety of ...
Serverless Execution of Scientific Workflows
Service-Oriented ComputingAbstractIn this paper, we present a serverless workflow execution system (DEWE v3) with Function-as-a-Service (FaaS aka serverless computing) as the target execution environment. DEWE v3 is designed to address problems of (1) execution of large-scale ...
Towards Efficient Mapping, Scheduling, and Execution of HPC Applications on Platforms in Cloud
IPDPSW '13: Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing Workshops and PhD ForumThe advantages of pay-as-you-go model, elasticity, and the flexibility and customization offered by virtualization make cloud computing an attractive option for meeting the needs of some High Performance Computing (HPC) users. However, there is a ...
Comments
Information & Contributors
Information
Published In
December 2023
502 pages
ISBN:9798400702341
DOI:10.1145/3603166
- Co-chairs:
- Omer Rana,
- Massimo Villari,
- Program Co-chairs:
- Song Fu,
- Lorenzo Carnevale
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 April 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
UCC '23
Sponsor:
UCC '23: IEEE/ACM 16th International Conference on Utility and Cloud Computing
December 4 - 7, 2023
Taormina (Messina), Italy
Acceptance Rates
Overall Acceptance Rate 38 of 125 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 113Total Downloads
- Downloads (Last 12 months)113
- Downloads (Last 6 weeks)13
Reflects downloads up to 23 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in