research-article

SZZ unleashed: an open implementation of the SZZ algorithm - featuring example usage in a study of just-in-time bug prediction for the Jenkins project

Authors:

Oscar Svensson,

Daniel HanssonAuthors Info & Claims

MaLTeSQuE 2019: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation

Pages 7 - 12

https://doi.org/10.1145/3340482.3342742

Published: 27 August 2019 Publication History

Abstract

Machine learning applications in software engineering often rely on detailed information about bugs. While issue trackers often contain information about when bugs were fixed, details about when they were introduced to the system are often absent. As a remedy, researchers often rely on the SZZ algorithm as a heuristic approach to identify bug-introducing software changes. Unfortunately, as reported in a recent systematic literature review, few researchers have made their SZZ implementations publicly available. Consequently, there is a risk that research effort is wasted as new projects based on SZZ output need to initially reimplement the approach. Furthermore, there is a risk that newly developed (closed source) SZZ implementations have not been properly tested, thus conducting research based on their output might introduce threats to validity. We present SZZ Unleashed, an open implementation of the SZZ algorithm for git repositories. This paper describes our implementation along with a usage example for the Jenkins project, and conclude with an illustrative study on just-in-time bug prediction. We hope to continue evolving SZZ Unleashed on GitHub, and warmly invite the community to contribute.

References

[1]

K. Berg and O. Svensson. 2018. SZZ Unleashed: Bug Prediction on the Jenkins Core Repository (Open Source Implementations of Bug Prediction Tools on Commit Level). Msc Thesis, Lund University, Sweden.

[2]

G. Canfora, L. Cerulo, and M. Di Penta. 2007. Identifying Changed Source Code Lines from Version Repositories. In Proc. of the 4th International Workshop on Mining Software Repositories.

Digital Library

[3]

Y. Cavalcanti, P. Silveira Neto, I. Machado, T. Vale, E. Almeida, and S. Meira. 2014. Challenges and Opportunities for Software Change Request Repositories: A Systematic Mapping Study. Journal of Software: Evolution and Process 26, 7 (2014), 620–653.

Digital Library

[4]

J. Correia. 2017. old-szz. https://github.com/intelligentagents/old-szz

[5]

J. Czerwonka, R. Das, N. Nagappan, A. Tarvo, and A. Teterev. 2011. CRANE: Failure Prediction, Change Analysis and Test Prioritization in Practice âĂŞ Experiences from Windows. In Proc. of the 4th Conference on Software Testing, Verification and Validation. 357–366.

Digital Library

[6]

M. D’Ambros, M. Lanza, and R. Robbes. 2009. On the Relationship Between Change Coupling and Software Defects. In 2009 16th Working Conference on Reverse Engineering. 135–144.

Digital Library

[7]

M. de Freitas Farias, R. Novais, M. Junior, L. da Silva Carvalho, M. Mendonca, and R. Spinola. 2016. A Systematic Mapping Study on Mining Software Repositories. In Proc. of the 31st Annual ACM Symposium on Applied Computing. 1472–1479.

Digital Library

[8]

E. Engström, P. Runeson, and M. Skoglund. 2010. A Systematic Review on Regression Test Selection Techniques. Information and Software Technology 52, 1 (2010), 14–30.

Digital Library

[9]

N. Fenton and M. Neil. 1999. A Critique of Software Defect Prediction Models. Transactions on Software Engineering 25, 5 (1999), 675–689. 1109/32.815326

Digital Library

[10]

M. Godfrey and L. Zou. 2005. Using Origin Analysis to Detect Merging and Splitting of Source Code Entities. Transactions on Software Engineering 31, 2 (2005), 166–181.

Digital Library

[11]

T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell. 2012. A Systematic Literature Review on Fault Prediction Performance in Software Engineering. Transactions on Software Engineering 38, 6 (2012), 1276–1304.

Digital Library

[12]

1109/TSE.2011.103

[13]

L. Jonsson, M. Borg, D. Broman, K. Sandahl, S. Eldh, and P. Runeson. 2016. Automated Bug Assignment: Ensemble-based Machine Learning in Large Scale Industrial Contexts. Empirical Software Engineering 21, 4 (2016), 1533–1578.

Digital Library

[14]

Y. Kamei, E. Shihab, B. Adams, A. Hassan, A. Mockus, A. Sinha, and N. Ubayashi. 2013. A Large-scale Empirical Study of Just-in-Time Quality Assurance. In Transactions on Software Engineering, Vol. 39. 757–773. TSE.2012.70

Digital Library

[15]

Sunghun Kim, Thomas Zimmermann, Kai Pan, and Whitehead James. 2006. Automatic identification of bug-introducing changes. In Proc. of the 21st International Conference on Automated Software Engineering. 81–90.

Digital Library

[16]

G. Lemaitre, F. Nogueira, and C. Aridas. 2017. Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning. Journal of Machine Learning Research 18, 17 (2017), 1–5.

Digital Library

[17]

R. Moser, W. Pedrycz, and G. Succi. 2008. A Comparative Analysis of the Efficiency of Change Metrics and Static Code Attributes for Defect Prediction. In Proc.of the 30th International Conference on Software Engineering. 181–190. org/10.1145/1368088.1368114

Digital Library

[18]

N. Nagappan and T. Ball. 2005. Use of Relative Code Churn Measures to Predict System Defect Density. In Proc. of the 27th International Conference on Software Engineering. 284–292.

Digital Library

[19]

F. Rahman, S. Khatri, E. Barr, and P. Devanbu. 2014. Comparing Static Bug Finders and Statistical Prediction. In Proc. of the 36th International Conference on Software Engineering. 424–434.

Digital Library

[20]

G. Rodriguez-Perez, G. Robles, and J. Gonzalez-Barahona. 2018. Reproducibility and Credibility in Empirical Software Engineering: A Case Study based on a Systematic Literature Review of the use of the SZZ algorithm. Information and Software Technology (2018).

[21]

C. Rosen, B. Grawi, and E. Shihab. 2015. Commit Guru: Analytics and Risk Prediction of Software Commits. In Proc. of the 2015 10th Joint Meeting on Foundations of Software Engineering. 966–969.

Digital Library

[22]

J. Sliwerski, T. Zimmermann, and A. Zeller. 2005. When Do Changes Induce Fixes?. In Proc. of the 2005 International Workshop on Mining Software Repositories, Vol. 30. 1–5.

Digital Library

[23]

M. Sohn, S. Pearce, A. Loskutov, C. Aniszczyk, C. Halstrick, C. Ranger, D. Borowitz, D. Pursehouse, G. Wagenknecht, J. Nieder, K. Sawicki, M. Kinzler, R. Rosenberg, R. Stocker, S. Zivkov, S. Lay, T. Parker, and T. Wolf. 2017. Eclipse JGit. https: //github.com/eclipse/jgit

[24]

O. Svensson and K. Berg. 2018. SZZ Unleashed. https://github.com/wogscpar/ SZZUnleashed

[25]

M. Tan, L. Tan, S. Dara, and C. Mayeux. 2015. Online Defect Prediction for Imbalanced Data. In Proc. of the 37th International Conference on Software Engineering. 99–108.

Digital Library

[26]

A. Tornhill. 2017. code-maat. https://github.com/adamtornhill/code-maat

[27]

C. Williams and J. Spacco. 2008. SZZ Revisited: Verifying When Changes Induce Fixes. In Proc. of the 2008 Workshop on Defects in Large Software Systems. 32–36.

Digital Library

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Olewicki DHabchi SAdams B(2024)An Empirical Study on Code Review Activity Prediction and Its Impact in PracticeProceedings of the ACM on Software Engineering10.1145/36608061:FSE(2238-2260)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660806
Lyu YKang HWidyasari RLawall JLo D(2024)Evaluating SZZ Implementations: An Empirical Study on the Linux KernelIEEE Transactions on Software Engineering10.1109/TSE.2024.340671850:9(2219-2239)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3406718
Show More Cited By

Index Terms

SZZ unleashed: an open implementation of the SZZ algorithm - featuring example usage in a study of just-in-time bug prediction for the Jenkins project
1. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Maintaining software
  2. Software notations and tools
    1. Software configuration management and version control systems
    2. Software maintenance tools

Recommendations

SZZ revisited: verifying when changes induce fixes
DEFECTS '08: Proceedings of the 2008 workshop on Defects in large software systems

Automatically identifying commits that induce fixes is an important task, as it enables researchers to quickly and efficiently validate many types of software engineering analyses, such as software metrics or models for predicting faulty components. ...
Evaluating SZZ Implementations Through a Developer-informed Oracle
ICSE '21: Proceedings of the 43rd International Conference on Software Engineering

The SZZ algorithm for identifying bug-inducing changes has been widely used to evaluate defect prediction techniques and to empirically investigate when, how, and by whom bugs are introduced. Over the years, researchers have proposed several heuristics ...
An empirical study on the use of SZZ for identifying inducing changes of non-functional bugs
Abstract
Non-functional bugs, e.g., performance bugs and security bugs, bear a heavy cost on both software developers and end-users. For example, IBM estimates the cost of a single data breach to be millions of dollars. Tools to reduce the occurrence, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MaLTeSQuE 2019: Proceedings of the 3rd ACM SIGSOFT International Workshop on Machine Learning Techniques for Software Quality Evaluation

August 2019

42 pages

ISBN:9781450368551

DOI:10.1145/3340482

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

ITEA3

Conference

ESEC/FSE '19

Sponsor:

SIGSOFT

ESEC/FSE '19: 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 27, 2019

Tallinn, Estonia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
494
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)15

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Olewicki DHabchi SAdams B(2024)An Empirical Study on Code Review Activity Prediction and Its Impact in PracticeProceedings of the ACM on Software Engineering10.1145/36608061:FSE(2238-2260)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660806
Lyu YKang HWidyasari RLawall JLo D(2024)Evaluating SZZ Implementations: An Empirical Study on the Linux KernelIEEE Transactions on Software Engineering10.1109/TSE.2024.340671850:9(2219-2239)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3406718
Watanabe HKondo MChoi EMizuno O(2024)Benefits and Pitfalls of Token-Level SZZ: An Empirical Study on OSS Projects2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00084(776-786)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00084
Zhao YLi XDeng WLi YGuo XTian QFan Y(2024)Fine-Grained Bug Localization Based on Rich Context using Attention Tree-GRU2024 5th International Conference on Computer Engineering and Application (ICCEA)10.1109/ICCEA62105.2024.10603974(640-646)Online publication date: 12-Apr-2024
https://doi.org/10.1109/ICCEA62105.2024.10603974
Suhag VDubey SSharma B(2024)Software defect prediction using global and local modelsInternational Journal of System Assurance Engineering and Management10.1007/s13198-024-02407-715:8(4003-4017)Online publication date: 2-Jul-2024
https://doi.org/10.1007/s13198-024-02407-7
Maes-Bermejo MSerebrenik AGallego MGortázar FRobles GGonzález Barahona J(2024)Hunting bugs: Towards an automated approach to identifying which change caused a bug through regression testingEmpirical Software Engineering10.1007/s10664-024-10479-z29:3Online publication date: 4-May-2024
https://doi.org/10.1007/s10664-024-10479-z
Bryan JMoriano P(2023)Graph-based machine learning improves just-in-time defect predictionPLOS ONE10.1371/journal.pone.028407718:4(e0284077)Online publication date: 13-Apr-2023
https://doi.org/10.1371/journal.pone.0284077
Reid DRahkema KWalden JMcIntosh SChoi EHerbold S(2023)Large Scale Study of Orphan Vulnerabilities in the Software Supply ChainProceedings of the 19th International Conference on Predictive Models and Data Analytics in Software Engineering10.1145/3617555.3617872(22-32)Online publication date: 8-Dec-2023
https://dl.acm.org/doi/10.1145/3617555.3617872
Liu SGuo ZLi YWang CChen LSun ZZhou YXu B(2023)Inconsistent Defect Labels: Essence, Causes, and InfluenceIEEE Transactions on Software Engineering10.1109/TSE.2022.315678749:2(586-610)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TSE.2022.3156787
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents