skip to main content
10.1145/1985374.1985386acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Different strokes for different folks: a case study on software metrics for different defect categories

Published: 24 May 2011 Publication History

Abstract

Defect prediction has been evolved with variety of metric sets, and defect types. Researchers found code, churn, and network metrics as significant indicators of defects. However, all metric sets may not be informative for all defect categories such that only one metric type may represent majority of a defect category. Our previous study showed that defect category sensitive prediction models are more successful than general models, since each category has different characteristics in terms of metrics. We extend our previous work, and propose specialized prediction models using churn, code, and network metrics with respect to three defect categories. Results show that churn metrics are the best for predicting all defects. The strength of correlation for code and network metrics varies with defect category: Network metrics have higher correlations than code metrics for defects reported during functional testing and in the field, and vice versa for defects reported during system testing.

References

[1]
Networkx website. http://networkx.lanl.gov/.
[2]
The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley, 1995.
[3]
Why projects fail: Nasa's mars climate orbiter project. Technical report, JSC Centre of Expertise in the Planning & Implementation of Information Systems, 2003.
[4]
B. Caglayan, A. Bener, and S. Koch. Merits of using repository metrics in defect prediction for open source projects. 2009 ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development, pages 31--36, May 2009.
[5]
B. Caglayan, A. Tosun, A. Miranskyy, A. Bener, and N. Ruffolo. Usage of multiple prediction models based on defect categories. In Proceedings of the 6th International Conference on Predictive Models in Software Engineering, PROMISE '10, pages 8:1--8:9, New York, NY, USA, 2010. ACM.
[6]
M. J. Germain. Can software kill you? TechnewsWorld, Technology Special Report, 2004.
[7]
Y. Jiang, B. Cukic, and T. Menzies. Can data transformation help in the detection of fault-prone modules? In DEFECTS '08: Proceedings of the 2008 workshop on Defects in large software systems, pages 16--20, New York, NY, USA, 2008. ACM.
[8]
E. Kocaguneli, A. Tosun, A. B. Bener, B. Turhan, and B. Caglayan. Prest: An intelligent software metrics extraction, analysis and defect prediction tool. In SEKE, pages 637--642, 2009.
[9]
S. Lessmann, B. Baesens, C. Mues, and S. Pietsch. Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Trans. Softw. Eng., 34(4):485--496, 2008.
[10]
M. A. Maloof. Learning when data sets are imbalanced and when costs are unequal and unknown. In ICML-2003 Workshop on Learning from Imbalanced Data Sets II, 2003.
[11]
A. Meneely, L. Williams, W. Snipes, and J. Osborne. Predicting failures with developer networks and social network analysis. In SIGSOFT '08/FSE-16: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pages 13--23, New York, NY, USA, 2008. ACM.
[12]
T. Menzies, J. Greenwald, and A. Frank. Data mining static code attributes to learn defect predictors. Software Engineering, IEEE Transactions on, 33(1):2--13, 2007.
[13]
T. Menzies, B. Turhan, A. Bener, G. Gay, B. Cukic, and Y. Jiang. Implications of ceiling effects in defect predictors. In Proceedings of the 4th international workshop on Predictor models in software engineering, PROMISE '08, pages 47--54, New York, NY, USA, 2008. ACM.
[14]
N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Proceedings of the 27th international conference on Software engineering, ICSE '05, pages 284--292, New York, NY, USA, 2005. ACM.
[15]
N. Nagappan and T. Ball. Using software dependencies and churn metrics to predict field failures: An empirical case study. In ESEM '07: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, pages 364--373, Washington, DC, USA, 2007. IEEE Computer Society.
[16]
N. Nagappan, T. Ball, and B. Murphy. Using historical in-process and product metrics for early estimation of software failures. In Proceedings of the 17th International Symposium on Software Reliability Engineering, pages 62--74, Washington, DC, USA, 2006. IEEE Computer Society.
[17]
N. Nagappan, B. Murphy, and V. Basili. The influence of organizational structure on software quality: an empirical case study. In ICSE '08: Proceedings of the 30th international conference on Software engineering, pages 521--530, New York, NY, USA, 2008. ACM.
[18]
T. J. Ostrand, E. J. Weyuker, and R. M. Bell. Predicting the location and number of faults in large software systems. IEEE Transactions on Software Engineering, 31(4):340--355, 2005.
[19]
T. J. Ostrand, E. J. Weyuker, and R. M. Bell. Automating algorithms for the identification of fault-prone files. pages --, 2007.
[20]
Y. Shin, R. Bell, T. Ostrand, and E. Weyuker. Does calling structure information improve the accuracy of fault prediction? In In Mining Software Repositories (MSR '09), 6th IEEE International Working Conference on (May 2009), pages 61--70, 2009.
[21]
A. Tosun, B. Turhan, and A. Bener. Practical considerations in deploying ai for defect prediction: a case study within the turkish telecommunication industry. In PROMISE '09: Proceedings of the 5th International Conference on Predictor Models in Software Engineering, pages 1--9, New York, NY, USA, 2009. ACM.
[22]
A. Tosun, B. Turhan, and A. Bener. Validation of network measures as indicators of defective modules in software systems. Proceedings of the 5th International Conference on Predictor Models in Software Engineering - PROMISE '09, page 1, 2009.
[23]
B. Turhan, T. Menzies, A. Bener, and J. Distefano. On the relative value of cross-company and within-company data for defect prediction. Empirical Software Engineering Journal, 2009. in print. DOI 10.1007/s10664-008-9103-7.
[24]
E. Weyuker, T. Ostrand, and R. Bell. Do too many cooks spoil the broth? using the number of developers to enhance defect prediction models. Empirical Software Engineering, 13(5):539--559, 2008.
[25]
T. Zimmermann and N. Nagappan. Predicting defects using network analysis on dependency graphs. In ICSE '08: Proceedings of the 30th international conference on Software engineering, pages 531--540, New York, NY, USA, 2008. ACM.
[26]
T. Zimmermann, R. Premraj, and A. Zeller. Predicting defects for eclipse. In PROMISE '07: Proceedings of the Third International Workshop on Predictor Models in Software Engineering, page 9, Washington, DC, USA, 2007. IEEE Computer Society.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WETSoM '11: Proceedings of the 2nd International Workshop on Emerging Trends in Software Metrics
May 2011
90 pages
ISBN:9781450305938
DOI:10.1145/1985374
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 May 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. churn metrics
  2. network metrics
  3. software defect prediction
  4. static code metrics

Qualifiers

  • Research-article

Conference

ICSE11
Sponsor:
ICSE11: International Conference on Software Engineering
May 24, 2011
HI, Waikiki, Honolulu, USA

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Evaluating complexity and digitizability of regulations and contracts for a blockchain application designProceedings of the 1st International Workshop on Emerging Trends in Software Engineering for Blockchain10.1145/3194113.3194117(25-29)Online publication date: 27-May-2018
  • (2018)An initial research on Halstead's technique for pattern similarity relationship study2018 International Conference on Information and Communications Technology (ICOIACT)10.1109/ICOIACT.2018.8350771(773-777)Online publication date: Mar-2018
  • (2018)Database enginesJournal of Software: Evolution and Process10.1002/smr.191530:4Online publication date: 17-Apr-2018
  • (2016)An Investigation of Essential Topics on Software Fault-Proneness Prediction2016 International Symposium on System and Software Reliability (ISSSR)10.1109/ISSSR.2016.016(37-46)Online publication date: Oct-2016
  • (2016)On Automatic Detection of Performance Bugs2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW.2016.43(132-139)Online publication date: Oct-2016
  • (2016)Studying high impact fix-inducing changesEmpirical Software Engineering10.1007/s10664-015-9370-z21:2(605-641)Online publication date: 1-Apr-2016
  • (2015)The Art and Science of Analyzing Software DataundefinedOnline publication date: 15-Sep-2015
  • (2014)An object based software tool for software measurementInternational Conference on Information Communication and Embedded Systems (ICICES2014)10.1109/ICICES.2014.7033772(1-5)Online publication date: Feb-2014
  • (2013)An algorithmic approach to missing data problem in modeling human aspects in software developmentProceedings of the 9th International Conference on Predictive Models in Software Engineering10.1145/2499393.2499398(1-10)Online publication date: 9-Oct-2013
  • (2013)Influence of confirmation biases of developers on software qualitySoftware Quality Journal10.1007/s11219-012-9180-021:2(377-416)Online publication date: 1-Jun-2013
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media