skip to main content
10.1145/3379177.3388905acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Emerging and Changing Tasks in the Development Process for Machine Learning Systems

Published: 16 September 2020 Publication History

Abstract

Integrating machine learning components in software systems is a task more and more companies are confronted with. However, there is not much knowledge today on how the software development process needs to change, when such components are integrated into a software system. We performed an interview study with 16 participants, focusing on emerging and changing task. The results uncover a set of 25 tasks associated to different software development phases, such as requirements engineering or deployment. We are just starting to understand the implications of using machine-learning components on the software development process. This study allows some first insights into how widespread the required process changes are.

References

[1]
Saleema Amershi, Andrew Begel, Christian Bird, Rob DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software Engineering for Machine Learning: A Case Study. XP 2019 (forthcoming) (2019).
[2]
Anders Arpteg, Björn Brinne, Luka Crnkovic-Friis, and Jan Bosch. 2018. Software Engineering Challenges of Deep Learning. CoRR abs/1810.12034 (2018). arXiv:1810.12034 http://arxiv.org/abs/1810.12034
[3]
Rob Ashmore, Radu Calinescu, and Colin Paterson. 2019. Assuring the machine learning lifecycle: Desiderata, methods, and challenges. arXiv preprint arXiv:1905.04223 (2019).
[4]
Julia Bailey. 2008. First steps in qualitative data analysis: transcribing. Family Practice 25, 2 (02 2008), 127--131. https://doi.org/10.1093/fampra/cmn003arXiv:http://oup.prod.sis.lan/fampra/article-pdf/25/2/127/1341812/cmn003.pdf
[5]
Elizamary de Souza Nascimento, Iftekhar Ahmed, Edson Oliveira, Márcio Piedade Palheta, Igor Steinmacher, and Tayana Conte. 2019. Understanding Development Process of Machine Learning Systems: Challenges and Solutions. In 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 1--6.
[6]
Miryung Kim, Thomas Zimmermann, Robert DeLine, and Andrew Begel. 2016. The Emerging Role of Data Scientists on Software Development Teams. In Proceedings of the 38th International Conference on Software Engineering (ICSE '16). ACM, New York, NY, USA, 96--107. https://doi.org/10.1145/2884781.2884783
[7]
Beatriz López, Natàlia Mordvanyuk, Pablo Gay, and Albert Pla. 2018. Knowledge Representation and Machine Learning on Wearable Sensor Data: A Study on Gait Monitoring. In Proceedings of the First International Conference on Data Science, E-learning and Information Systems (DATA '18). ACM, New York, NY, USA, Article 45, 2 pages. https://doi.org/10.1145/3279996.3280041
[8]
Lucy Ellen Lwakatare, Aiswarya Raj, Jan Bosch, Helena Holmström Olsson, and Ivica Crnkovic. 2019. A taxonomy of software engineering challenges for machine learning systems: An empirical investigation. XP 2019 (forthcoming) (2019).
[9]
Tim Menzies and Mark Rogers. 2015. Sharing data and models in software engineering. [electronic resource]. Waltham, Massachusetts: Morgan Kaufmann, 2015.
[10]
Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang, and Martin Zinkevich. 2017. Data management challenges in production machine learning. In Proceedings of the 2017 ACM International Conference on Management of Data. ACM, 1723--1726.
[11]
Per Runeson and Martin Höst. 2009. Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empirical Softw. Engg. 14, 2 (April 2009), 131--164. https://doi.org/10.1007/s10664-008-9102-8
[12]
Per Runeson, Martin Host, Austen Rainer, and Bjorn Regnell. 2012. Case study research in software engineering: Guidelines and examples. John Wiley & Sons.
[13]
Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. 2017. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296 (2017).
[14]
Carlton E Sapp. 2017. Preparing and architecting for machine learning. Gartner Technical Professional Advice (2017), 1--37.
[15]
David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. In Advances in neural information processing systems. 2503--2511.
[16]
Mojtaba Vaismoradi, Hannele Turunen, and Terese Bondas. 2013. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences 15, 3 (2013), 398--405.
[17]
Zhiyuan Wan, Xin Xia, David Lo, and Gail C Murphy. 2019. How does Machine Learning Change Software Development Practices? IEEE Transactions on Software Engineering (2019).
[18]
Hironori Washizaki, Hiromu Uchida, Foutse Khomh, and Yann-Gaël Guéhéneuc. 2019. Studying software engineering patterns for designing machine learning systems. In 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE, 49--495.
[19]
Yasuhiro Watanabe, Hironori Washizaki, Kazunori Sakamoto, Daisuke Saito, Kiyoshi Honda, Naohiko Tsuda, Yoshiaki Fukazawa, and Nobukazu Yoshioka. 2019. Preliminary Systematic Literature Review of Machine Learning System Development Process. arXiv preprint arXiv:1910.05528 (2019).
[20]
Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R Lyu, and Miryung Kim. 2019. An empirical study of common challenges in developing deep learning applications. In The 30th IEEE International Symposium on Software Reliability Engineering (ISSRE).
[21]
Luisa M Zintgraf, Taco S Cohen, Tameem Adel, and Max Welling. 2017. Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint arXiv:1702.04595 (2017).

Cited By

View all

Index Terms

  1. Emerging and Changing Tasks in the Development Process for Machine Learning Systems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICSSP '20: Proceedings of the International Conference on Software and System Processes
    June 2020
    208 pages
    ISBN:9781450375122
    DOI:10.1145/3379177
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 September 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Challenges
    2. Machine learning
    3. Roles
    4. Software process

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICSSP '20
    Sponsor:

    Upcoming Conference

    ICSE 2025

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)37
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 31 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media