research-article

Efficient operating system scheduling for performance-asymmetric multi-core architectures

Authors:

Dan Baumberger,

David A. Koufaty,

Scott HahnAuthors Info & Claims

SC '07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing

Article No.: 53, Pages 1 - 11

https://doi.org/10.1145/1362622.1362694

Published: 10 November 2007 Publication History

Abstract

Recent research advocates asymmetric multi-core architectures, where cores in the same processor can have different performance. These architectures support single-threaded performance and multithreaded throughput at lower costs (e.g., die size and power). However, they also pose unique challenges to operating systems, which traditionally assume homogeneous hardware. This paper presents AMPS, an operating system scheduler that efficiently supports both SMP-and NUMA-style performance-asymmetric architectures. AMPS contains three components: asymmetry-aware load balancing, faster-core-first scheduling, and NUMA-aware migration. We have implemented AMPS in Linux kernel 2.6.16 and used CPU clock modulation to emulate performance asymmetry on an SMP and NUMA system. For various workloads, we show that AMPS achieves a median speedup of 1.16 with a maximum of 1.44 over stock Linux on the SMP, and a median of 1.07 with a maximum of 2.61 on the NUMA system. Our results also show that AMPS improves fairness and repeatability of application performance measurements.

References

[1]

J. B. Andrews and C. D. Polychronopoulos. An analytical approach to performance/cost modeling of parallel computers. Journal of Parallel and Distributed Computing, 12(4):343--356, Aug. 1991.

Digital Library

[2]

M. Annavaram, E. Grochowski, and J. Shen. Mitigating Amdahl's law through EPI throttling. In Proceedings of the 32nd Annual International Symposium on Computer Architecture, pages 298--309, June 2005.

Digital Library

[3]

S. Balakrishnan, R. Rajwar, M. Upton, and K. Lai. The impact of performance asymmetry in emerging multicore architectures. In Proceedings of the 32nd Annual International Symposium on Computer Architecture, pages 506--517, June 2005.

Digital Library

[4]

C. Banino, O. Beaumont, L. Carter, J. Ferrante, A. Legrand, and Y. Robert. Scheduling strategies for master-slave tasking on heterogeneous processor platforms. IEEE Transactions on Parallel and Distributed Systems, 15(4):319--330, Apr. 2004.

Digital Library

[5]

M. A. Bender and M. O. Rabin. Scheduling Cilk multithreaded parallel programs on processors of different speeds. In Proceedings of the Twelfth ACM Symposium on Parallel Algorithms and Architectures, pages 13--21, July 2000.

Digital Library

[6]

S. Y. Borkar, P. Dubey, K. C. Kahn, D. J. Kuck, H. Mulder, S. S. Pawlowski, and J. Rattner. Platform 2015: Intel® processor and platform evolution for the next decade. White Paper, Intel Corporation, 2005.

[7]

R. Chandra, S. Devine, B. Verghese, A. Gupta, and M. Rosenblum. Scheduling and page migration for multiprocessor compute servers. In Proceedings of the Sixth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 12--24, Oct. 1994.

Digital Library

[8]

M. DeVuyst, R. Kumar, and D. M. Tullsen. Exploiting unbalanced thread scheduling for energy and performance on a CMP of SMT processors. In Proceedings of the 20th International Parallel and Distributed Processing Symposium, Apr. 2006.

Digital Library

[9]

A. Fedorova, M. Seltzer, C. Small, and D. Nussbaum. Performance of multithreaded chip multiprocessors and implications for operating system design. In Proceedings of the 2005 USENIX Annual Technical Conference, pages 395--398, Apr. 2005.

Digital Library

[10]

R. J. O. Figueiredo and J. A. B. Fortes. Impact of heterogeneity on DSM performance. In Proceedings of the Sixth IEEE Symposium on High-Performance Computer Architecture, pages 26--35, Jan. 2000.

[11]

S. Ghiasi, T. Keller, and F. Rawson. Scheduling for heterogeneous processors in server systems. In Proceedings of the 2nd Conference on Computing Frontiers, pages 199--210, May 2005.

Digital Library

[12]

R. A. Hankins, G. N. Chinya, J. D. Collins, P. H. Wang, R. Rakvic, H. Wang, and J. P. Shen. Multiple instruction stream processor. In Proceedings of the 33rd Annual International Symposium on Computer Architecture, pages 114--127, June 2006.

Digital Library

[13]

R. Kumar, K. I. Farkas, N. P. Jouppi, P. Ranganathan, and D. M. Tullsen. Single-ISA heterogeneous multi-core architectures: The potential for processor power reduction. In Proceedings of the 36th Annual IEEE/ACM International Symposium on Microarchitecture, pages 81--92, Dec. 2003.

Digital Library

[14]

R. Kumar, D. M. Tullsen, N. P. Jouppi, and P. Ranganathan. Heterogeneous chip multiprocessing. IEEE Computer, 38(11):32--38, Nov. 2005.

Digital Library

[15]

R. Kumar, D. M. Tullsen, P. Ranganathan, N. P. Jouppi, and K. I. Farkas. Single-ISA heterogeneous multi-core architectures for multithreaded workload performance. In Proceedings of the 31st Annual International Symposium on Computer Architecture, pages 64--75, June 2004.

Digital Library

[16]

R. P. LaRowe, Jr., C. S. Ellis, and L. S. Kaplan. The robustness of NUMA memory management. In Proceedings of the 13th ACM Symposium on Operating System Principles, pages 137--151, Oct. 1991.

Digital Library

[17]

Linux Kernel Mailing List. Scalability of signal delivery for POSIX threads. http://lkml.org/lkml/2004/11/22/432, Nov. 2004.

[18]

D. Menascé and V. Almeida. Cost-performance analysis of heterogeneity in supercomputer architectures. In Proceedings of the 1990 International Conference on Supercomputing, pages 169--177, June 1990.

Digital Library

[19]

D. Pham, S. Asano, M. Bolliger, M. N. Day, H. P. Hofstee, C. Johns, J. Kahle, A. Kameyama, J. Keaty, Y. Masubuchi, M. Riley, D. Shippy, D. Stasiak, M. Suzuoki, M. Wang, J. Warnock, S. Weitzel, D. Wendel, T. Yamazaki, and K. Yazawa. The design and implementation of a first generation CELL* processor. In IEEE International Solid-State Circuits Conference Digest of Technical Papers, pages 184--185, Feb. 2005.

[20]

M. D. Powell, M. Gomaa, and T. Vijaykumar. Heat-and-run: Leveraging SMT and CMP to manage power density through the operating system. In Proceedings of the Eleventh International Conference on Architectural Support for Programming Languages and Operating Systems, pages 260--270, Oct. 2004.

Digital Library

[21]

G. C. Sih and E. A. Lee. A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures. IEEE Transactions on Parallel and Distributed Systems, 4(2):175--187, Feb. 1993.

Digital Library

[22]

A. Snavely and D. M. Tullsen. Symbiotic jobscheduling for a simultaneous multithreading processor. In Proceedings of the Ninth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 234--244, Nov. 2000.

Digital Library

[23]

H. Topcuoglu, S. Hariri, and M.-Y. Wu. Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Transactions on Parallel and Distributed Systems, 13(3):260--274, Mar. 2002.

Digital Library

[24]

V. Uhlig, J. LeVasseur, E. Skoglund, and U. Dannowski. Towards scalable multiprocessor virtual machines. In Proceedings of the 3rd Virtual Machine Research and Technology Symposium, pages 43--56, May 2004.

Digital Library

[25]

H. Zheng and J. Nieh. SWAP: A scheduler with automatic process dependency detection. In Proceedings of the First Symposium on Networked Systems Design and Implementation, pages 183--196, Mar. 2004.

Digital Library

Cited By

Kim SZhao JAsanovic KNikolic BShao Y(2023)AuRORA: Virtualized Accelerator Orchestration for Multi-Tenant WorkloadsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614280(62-76)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614280
Liang QHanafy WBashir NIrwin DShenoy PSha KBanerjee SChen J(2023)Energy Time Fairness: Balancing Fair Allocation of Energy and Time for GPU WorkloadsProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3628435(53-66)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3628435
Xu YQian RWang YHuo Q(2023)iNUMAlloc: Towards Intelligent Memory Allocation for AI Accelerators with NUMA2023 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00155(929-936)Online publication date: 21-Dec-2023
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00155
Show More Cited By

Efficient operating system scheduling for performance-asymmetric multi-core architectures

Recommendations

Virtualizing performance asymmetric multi-core systems
ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

Performance-asymmetric multi-cores consist of heterogeneous cores, which support the same ISA, but have different computing capabilities. To maximize the throughput of asymmetric multi-core systems, operating systems are responsible for scheduling ...
Evaluating multi-core and many-core architectures through accelerating the three-dimensional Lax-Wendroff correction stencil

Wave propagation forward modeling is a widely used computational method in oil and gas exploration. The iterative stencil loops in such problems have broad applications in scientific computing. However, executing such loops can be highly time-consuming, ...
Accelerating critical section execution with asymmetric multi-core architectures
ASPLOS 2009

To improve the performance of a single application on Chip Multiprocessors (CMPs), the application must be split into threads which execute concurrently on multiple cores. In multi-threaded applications, critical sections are used to ensure that only ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing

November 2007

723 pages

ISBN:9781595937643

DOI:10.1145/1362622

General Chair:
Becky Verastegui
Oak Ridge National Laboratory

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SC '07

Sponsor:

SIGARCH
IEEE-CS

SC '07: International Conference for High Performance Computing, Networking, Storage and Analysis

November 10 - 16, 2007

Nevada, Reno

Acceptance Rates

SC '07 Paper Acceptance Rate 54 of 268 submissions, 20%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

188
Total Citations
View Citations
2,701
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)6

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim SZhao JAsanovic KNikolic BShao Y(2023)AuRORA: Virtualized Accelerator Orchestration for Multi-Tenant WorkloadsProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture10.1145/3613424.3614280(62-76)Online publication date: 28-Oct-2023
https://dl.acm.org/doi/10.1145/3613424.3614280
Liang QHanafy WBashir NIrwin DShenoy PSha KBanerjee SChen J(2023)Energy Time Fairness: Balancing Fair Allocation of Energy and Time for GPU WorkloadsProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3628435(53-66)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3628435
Xu YQian RWang YHuo Q(2023)iNUMAlloc: Towards Intelligent Memory Allocation for AI Accelerators with NUMA2023 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00155(929-936)Online publication date: 21-Dec-2023
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00155
Chen YQiu KChen LJia HZhang YXiao LLiu L(2022)Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systemsCCF Transactions on High Performance Computing10.1007/s42514-022-00110-24:4(394-406)Online publication date: 11-Oct-2022
https://doi.org/10.1007/s42514-022-00110-2
Fei PNa XZhi M(2022)A Novel Message Communication Mechanism Based on Partition NameSignal and Information Processing, Networking and Computers10.1007/978-981-19-3387-5_91(771-778)Online publication date: 2-Jul-2022
https://doi.org/10.1007/978-981-19-3387-5_91
Chiang MSu W(2021)Thread-Aware Mechanism to Enhance Inter-Node Load Balancing for Multithreaded Applications on NUMA SystemsApplied Sciences10.3390/app1114648611:14(6486)Online publication date: 14-Jul-2021
https://doi.org/10.3390/app11146486
Yu TZhong RJanjic VPetoumenos PZhai JLeather HThomson J(2021)Collaborative Heterogeneity-Aware OS Scheduler for Asymmetric Multicore ProcessorsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2020.304527932:5(1224-1237)Online publication date: 1-May-2021
https://doi.org/10.1109/TPDS.2020.3045279
Tan QXiao KHe WLei PChen L(2021)A Global Dynamic Load Balancing Mechanism with Low Latency for Micokernel Operating System2021 7th International Symposium on System and Software Reliability (ISSSR)10.1109/ISSSR53171.2021.00026(178-187)Online publication date: Sep-2021
https://doi.org/10.1109/ISSSR53171.2021.00026
Shahid AFahad MManumachu RLastovetsky A(2021)Energy Predictive Models of Computing: Theory, Practical Implications and Experimental Analysis on Multicore ProcessorsIEEE Access10.1109/ACCESS.2021.30751399(63149-63172)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3075139
Choudhury ANur K(2020)Qalitative Study of Contention-aware Scheduling Algorithm for Asymmetric Multicore ProcessorsProceedings of the International Conference on Computing Advancements10.1145/3377049.3377135(1-6)Online publication date: 10-Jan-2020
https://dl.acm.org/doi/10.1145/3377049.3377135
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten