Article

Towards an automated approach to use expert systems in the performance testing of distributed systems

Authors:

A. Omar Portillo-Dominguez,

Peter F. Sweeney,

Erik AltmanAuthors Info & Claims

JAMAICA 2014: Proceedings of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing

Pages 22 - 27

https://doi.org/10.1145/2631890.2631895

Published: 21 July 2014 Publication History

Abstract

Performance testing in distributed environments is challenging. Specifically, the identification of performance issues and their root causes are time-consuming and complex tasks which heavily rely on expertise. To simplify these tasks, many researchers have been developing tools with built-in expertise. However limitations exist in these tools, such as managing huge volumes of distributed data, that prevent their efficient usage for performance testing of highly distributed environments. To address these limitations, this paper presents an adaptive framework to automate the usage of expert systems in performance testing. Our validation assessed the accuracy of the framework and the time savings that it brings to testers. The results proved the benefits of the framework by achieving a significant decrease in the time invested in performance analysis and testing.

References

[1]

E. Altman, M. Arnold, S. Fink, and N. Mitchell. Performance analysis of idle programs. ACM SIGPLAN Notices, Oct. 2010.

Digital Library

[2]

V. Angelopoulos, T. Parsons, J. Murphy, and P. O’Sullivan. GcLite: An Expert Tool for Analyzing Garbage Collection Behavior. ACSACW, 2012.

Digital Library

[3]

P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using magpie for request extraction and workload modelling. OSDI, 2004.

Digital Library

[4]

S. Chen, D. Moreland, S. Nepal, and J. Zic. Yet Another Performance Testing Framework. Australian Conference on Software Engineering, 2008.

Digital Library

[5]

Compuware. Applied Performance Management Survey. 2007.

[6]

S. Dosinger, Stefan, Richard Mordinyi. Communicating CI servers for increasing effectiveness of automated testing. ASE, 2012.

Digital Library

[7]

E. G. Jin, L. Song. Understanding and detecting real-world performance bugs. PLDI, 2012.

Digital Library

[8]

V. Hoorn, M. Rohr, W. Hasselbring, J. Waller, J. Ehlers, and D. Kieselhorst. Continuous Monitoring of Software Services: Design and Application of the Kieker Framework. 2009.

[9]

D. J. Kephart. The vision of autonomic computing. Computer, Jan. 2003.

Digital Library

[10]

E. Li, Mengchen. Dynamically Validating Static Memory Leak Warnings. ISSTA, 2013.

Digital Library

[11]

J. M. S. Bayan. Automatic stress and load testing for embedded systems. ICSA, 2006.

Digital Library

[12]

L. M. Salehie. Self-adaptive software: Landscape and research challenges. TAAS, 2009.

Digital Library

[13]

S. Shahamiri, W. Kadir, and S. Mohd-Hashim. A Comparative Study on Automated Software Test Oracle Methods. ICSEA, 2009.

Digital Library

[14]

M. Woodside, G. Franks, and D. C. Petriu. The Future of Software Performance Engineering. FOSE, 2007.

Digital Library

[15]

W. Xingen Wang, Bo Zhou. Model-based load testing of web applications. Journal of the Chinese Institute of Engineers, 2013.

[16]

E. Yu, Kai. Practical isolation of failure-inducing changes for debugging regression faults. ASE, 2012.

Digital Library

Cited By

Whiting EDatta S(2022)Design and Development of a Technology-Agnostic NFR Testing FrameworkProceedings of the 2022 5th International Conference on Software Engineering and Information Management10.1145/3520084.3520092(45-50)Online publication date: 21-Jan-2022
https://dl.acm.org/doi/10.1145/3520084.3520092
Ayala-Rivera VKaczmarski MMurphy JDarisa APortillo-Dominguez AWolter KKnottenbelt Wvan Hoorn ANambiar MKoziolek H(2018)One Size Does Not Fit AllProceedings of the 2018 ACM/SPEC International Conference on Performance Engineering10.1145/3184407.3184418(211-222)Online publication date: 30-Mar-2018
https://dl.acm.org/doi/10.1145/3184407.3184418
Ferme VPautasso CWolter KKnottenbelt Wvan Hoorn ANambiar MKoziolek H(2018)A Declarative Approach for Performance Tests Execution in Continuous Software Development EnvironmentsProceedings of the 2018 ACM/SPEC International Conference on Performance Engineering10.1145/3184407.3184417(261-272)Online publication date: 30-Mar-2018
https://dl.acm.org/doi/10.1145/3184407.3184417
Show More Cited By

Index Terms

Towards an automated approach to use expert systems in the performance testing of distributed systems
1. Hardware
  1. Robustness
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Performance metamorphic testing: motivation and challenges
ICSE-NIER '17: Proceedings of the 39th International Conference on Software Engineering: New Ideas and Emerging Results Track

Performance testing is a challenging task mainly due to the lack of test oracles, that is, mechanisms to decide whether the performance of a program under a certain workload is either acceptable or poor due to a performance bug. Metamorphic testing ...
Automated GUI performance testing

A significant body of prior work has devised approaches for automating the functional testing of interactive applications. However, little work exists for automatically testing their performance. Performance testing imposes additional requirements upon ...
Towards virtualized and automated software performance test architecture

In this paper, we propose the towards virtualized and automated software performance test architecture. In general, test engineers use the public performance testwares such as Load Runner, Silk Performer to validate the performance efficiency of their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

JAMAICA 2014: Proceedings of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing

July 2014

33 pages

ISBN:9781450329330

DOI:10.1145/2631890

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ISSTA '14

Sponsor:

SIGSOFT

ISSTA '14: International Symposium on Software Testing and Analysis

July 21, 2014

CA, San Jose, USA

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Whiting EDatta S(2022)Design and Development of a Technology-Agnostic NFR Testing FrameworkProceedings of the 2022 5th International Conference on Software Engineering and Information Management10.1145/3520084.3520092(45-50)Online publication date: 21-Jan-2022
https://dl.acm.org/doi/10.1145/3520084.3520092
Ayala-Rivera VKaczmarski MMurphy JDarisa APortillo-Dominguez AWolter KKnottenbelt Wvan Hoorn ANambiar MKoziolek H(2018)One Size Does Not Fit AllProceedings of the 2018 ACM/SPEC International Conference on Performance Engineering10.1145/3184407.3184418(211-222)Online publication date: 30-Mar-2018
https://dl.acm.org/doi/10.1145/3184407.3184418
Ferme VPautasso CWolter KKnottenbelt Wvan Hoorn ANambiar MKoziolek H(2018)A Declarative Approach for Performance Tests Execution in Continuous Software Development EnvironmentsProceedings of the 2018 ACM/SPEC International Conference on Performance Engineering10.1145/3184407.3184417(261-272)Online publication date: 30-Mar-2018
https://dl.acm.org/doi/10.1145/3184407.3184417
Kaczmarski MPerry PMurphy JPortillo-Dominguez ABinder WCortellessa VKoziolek ASmirni EPoess M(2017)In-Test Adaptation of Workload in Enterprise Application Performance TestingProceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion10.1145/3053600.3053614(69-72)Online publication date: 18-Apr-2017
https://dl.acm.org/doi/10.1145/3053600.3053614
Portillo‐Dominguez APerry PMagoni DMurphy J(2017)PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systemsSoftware: Practice and Experience10.1002/spe.250047:11(1837-1874)Online publication date: 11-Apr-2017
https://doi.org/10.1002/spe.2500

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents