Jul 14, 2020 · In this paper, we propose and develop a middleware Dagon, which leverages the complexity and heterogeneity of DAGs to jointly execute task scheduling and cache ...
In this work, we aim for joint DAG-aware task scheduling and cache management. We propose and develop a middleware. Dagon, which aims to improve resource ...
In this paper, we propose and develop a middleware Dagon, which leverages the complexity and heterogeneity of DAGs to jointly execute task scheduling and cache ...
Compared with the MapReduce [6][7][8] computing framework, the memory-based computing speed of Spark is more than one hundred times faster than the former, ...
Cache management policies that are designed for Spark exhibit poor performance in DAG-aware task-scheduling algorithms, which leads to cache misses and ...
Missing: Joint | Show results with:Joint
Mar 12, 2024 · DAG-Aware Joint Task Scheduling and Cache Management in Spark Clusters. Conference Paper. May 2020. Yinggen Xu · Liu Liu · Zhijun Ding · View.
DAG-aware harmonizing job scheduling and data caching for ...
dl.acm.org › doi › j.future.2024.03.005
Jul 18, 2024 · Existing caching and prefetching solutions utilize high-level information from data analytics frameworks to forecast future data accesses. They ...
People also ask
Which is responsible for task scheduling and memory management in spark?
What is task scheduling in spark?
An EATPCBFD scheduling algorithm based on clustering performance is proposed. Optimize the energy efficiency and SLA in Spark's native scheduling strategy.
A data-parallel job is characterized as a directed acyclic graph (DAG) which usually consists of multiple computation stages and across-stage data transfers ...
Missing: Joint | Show results with:Joint
A new cache management policy known as Long-Running Stage Set First (LSF) is proposed, which makes full use of the task dependencies to optimize the cache ...
Missing: Joint | Show results with:Joint
Amazon EMR Supports Workloads Based on Apache Spark, Apache Hive, Presto and Apache Hbase.