Article

Software testing: a research travelogue (2000–2014)

Authors:

Alessandro Orso,

Gregg RothermelAuthors Info & Claims

FOSE 2014: Future of Software Engineering Proceedings

Pages 117 - 132

https://doi.org/10.1145/2593882.2593885

Published: 31 May 2014 Publication History

Abstract

Despite decades of work by researchers and practitioners on numerous software quality assurance techniques, testing remains one of the most widely practiced and studied approaches for assessing and improving software quality. Our goal, in this paper, is to provide an accounting of some of the most successful research performed in software testing since the year 2000, and to present what appear to be some of the most significant challenges and opportunities in this area. To be more inclusive in this effort, and to go beyond our own personal opinions and biases, we began by contacting over 50 of our colleagues who are active in the testing research area, and asked them what they believed were (1) the most significant contributions to software testing since 2000 and (2) the greatest open challenges and opportunities for future research in this area. While our colleagues’ input (consisting of about 30 responses) helped guide our choice of topics to cover and ultimately the writing of this paper, we by no means claim that our paper represents all the relevant and noteworthy research performed in the area of software testing in the time period considered—a task that would require far more space and time than we have available. Nevertheless, we hope that the approach we followed helps this paper better reflect not only our views, but also those of the software testing community in general.

References

[1]

K. Adamopoulos, M. Harman, and R. M. Hierons. How to overcome the equivalent mutant problem and achieve tailored selective mutation using co-evolution. In GECCO (2), Volume 3103 of Lecture Notes In Computer Scienc, pages 1338–1349. Springer, 2004.

[2]

W. Afzal, R. Torkar, and R. Feldt. A systematic review of search-based testing for non-functional system properties. Information and Software Technology, 51(6):957–976, June 2009.

Digital Library

[3]

S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar-Walawege. A systematic review of the application and empirical investigation of search-based test case generation. IEEE Transactions on Software Engineering, 36(6):742–762, Nov. 2010.

Digital Library

[4]

N. Alshahwan and M. Harman. Automated web application testing using search based software engineering. In Proceedings of Automated Software Engineering, pages 3–12, 2011.

Digital Library

[5]

D. Amalfitano, A. R. Fasolino, and P. Tramontana. A GUI crawling-based technique for Android mobile application testing. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation Workshops, pages 252–261, 2011.

Digital Library

[6]

S. Anand, E. K. Burke, T. Y. Chen, J. Clark, M. B. Cohen, W. Grieskamp, M. Harman, M. J. Harrold, and P. Mcminn. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 86(8):1978–2001, Aug. 2013.

Digital Library

[7]

S. Anand, M. Naik, M. J. Harrold, and H. Yang. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 59:1–59:11, 2012.

Digital Library

[8]

J. H. Andrews, L. C. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? In Proceedings of the International Conference on Software Engineering, pages 402–411, May 2005.

Digital Library

[9]

Mit app inventor. http://appinventor.mit.edu/explore/.

[10]

A. Arcuri and L. Briand. Adaptive random testing: An illusion of effectiveness? In Proceedings of the International Symposium on Software Testing and Analysis, pages 265–275, July 2011.

Digital Library

[11]

A. Arcuri and X. Yao. Coevolving programs and unit tests from their specification. In In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, pages 397–400, 2007.

Digital Library

[12]

S. Artzi, J. Dolby, S. H. Jensen, A. Møller, and F. Tip. A framework for automated testing of Javascript web applications. In Proceedings of the International Conference on Software Engineering, pages 571–580, 2011.

Digital Library

[13]

Atlassian. Atlassian software systems: Bamboo. https://www.atlassian.com/software/bamboo.

[14]

A. I. Baars, M. Harman, Y. Hassoun, K. Lakhotia, P. McMinn, P. Tonella, and T. E. J. Vos. Symbolic search-based testing. In Proceedings of the International Conference on Automated Softwaare Engineering, pages 53–62, 2011.

Digital Library

[15]

J. Bailey and P. J. Stuckey. Discovery of minimal unsatisfiable subsets of constraints using hitting set dualization. In Proceedings of the International Conference on Practical Aspects of Declarative Languages, pages 174–186, 2005.

Digital Library

[16]

A. Baresel, D. Binkley, M. Harman, and B. Korel. Evolutionary testing in the presence of loop-assigned flags: A testability transformation approach. In Proceedings of the International Symposium on Software Testing and Analysis, pages 108–118, 2004.

Digital Library

[17]

K. Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2000.

Digital Library

[18]

N. E. Beckman, A. V. Nori, S. K. Rajamani, and R. J. Simmons. Proofs from tests. In Proceedings of the 2008 International Symposium on Software Testing and Analysis, pages 3–14, 2008.

Digital Library

[19]

A. Bertolino. Software testing research: Achievements, challenges, dreams. In 2007 Future of Software Engineering, pages 85–103, 2007.

Digital Library

[20]

A. Bertolino, G. DeAngelis, and A. Sabetta. VCR: Virtual capture and replay for performance testing. In Proceedings of Automated Software Engineering, pages 399–402, Nov. 2008.

Digital Library

[21]

R. V. Binder. Open source tools for model-based testing. http://robertvbinder.com/open-source-tools-for-model-basedtesting.

[22]

D. Binkley. Semantics guided regression test cost reduction. IEEE Transactions on Software Engineering, 23(8), Aug. 1997.

Digital Library

[23]

S. Biswas, R. Mall, M. Satpathy, and S. Sukumaran. Regression test selection techniques: A survey. Informatica, 35:289–321, 2011.

[24]

M. Borges, A. Filieri, M. d’Amorim, C. Pasareanu, and W. Visser. Compositional solution space quantification for probabilistic software analysis. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, 2014.

Digital Library

[25]

J. Bowring, A. Orso, and M. J. Harrold. Monitoring deployed software using software tomography. In Proceedings of the ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, pages 2–8, Nov. 2002.

Digital Library

[26]

R. C. Bryce and C. J. Colbourn. Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information and Software Technology, 48(10):960–970, Oct. 2006.

[27]

S. Bucur, V. Ureche, C. Zamfir, and G. Candea. Parallel symbolic execution for automated real-world software testing. In Proceedings of the Sixth Conference on Computer Systems, pages 183–198, 2011.

Digital Library

[28]

C. Cadar. KLEE-related publications and systems. http://klee.github.io/klee/Publications.html, 2014.

[29]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the USENIX Conference on Operating Systems Design and Implementation, pages 209–224, 2008.

Digital Library

[30]

C. Cadar and D. R. Engler. Execution generated test cases: How to make systems code crash itself. In Proceedings of the SPIN Symposium on Model Checking of Software, pages 2–23, 2005.

Digital Library

[31]

C. Cadar, P. Godefroid, S. Khurshid, C. S. Păsăreanu, K. Sen, N. Tillmann, and W. Visser. Symbolic execution for software testing in practice: Preliminary assessment. In Proceedings of the International Conference on Software Engineering, pages 1066–1071, 2011.

Digital Library

[32]

K.-P. Chan, T. Y. Chen, and D. Towey. Forgetting test cases. In Proceedings of the International Computer Software and Applications Conference, volume 1, pages 485–494, 2006.

Digital Library

[33]

N. Chen and S. Kim. Puzzle-based automatic testing: Bringing humans into the loop by solving puzzles. In Proceedings of the International Conference on Automated Software Engineering, pages 140–149, 2012.

Digital Library

[34]

T. Y. Chen, F.-C. Kuo, R. Merkel, and S. P. Ng. Mirror adaptive random testing. Information and Software Technology, 46(15):1001–1010, 2004.

[35]

T. Y. Chen, T. H. Tse, and Y. T. Yu. Proportional sampling strategy: A compendium and some insights. Journal of Systems and Software, 58(1):65–81, 2001.

Digital Library

[36]

Y. Chen, A. Groce, C. Zhang, W.-K. Wong, X. Fern, E. Eide, and J. Regehr. Taming compiler fuzzers. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 197–208, 2013.

Digital Library

[37]

V. Chipounov, V. Georgescu, C. Zamfir, and G. Candea. Selective symbolic execution. In Workshop on Hot Topics in Dependable Systems (HotDep), 2009.

[38]

L. A. Clarke and D. J. Richardson. Applications of symbolic evaluation. Journal of Systems and Software, 5(1):15–35, 1985.

Digital Library

[39]

J. Clause and A. Orso. A technique for enabling and supporting debugging of field failures. In Proceedings of the International Conference on Software Engineering, pages 261–270, 2007.

Digital Library

[40]

D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton. The AETG system: An approach to testing based on combinatorial design. IEEE Transactions on Software Engineering, 23(7):437–444, July 1997.

Digital Library

[41]

M. B. Cohen, M. B. Dwyer, and J. Shi. Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Transactions on Software Engineering, 34(5):633–650, Sept. 2008.

Digital Library

[42]

M. B. Cohen, P. B. Gibbons, W. B. Mugridge, and C. J. Colbourn. Constructing test suites for interaction testing. In Proceedings of the International Conference on Software Engineering, pages 38–48, 2003.

Digital Library

[43]

C. Colbourn. Combinatorial aspects of covering arrays. Le Matematich (Catania), 58:121–157, 2004.

[44]

C. J. Colbourn and M. B. Cohen. A deterministic density algorithm for pairwise interaction coverage. In Proceedings of the IASTED International Conference on Software Engineering, pages 242–252, 2004.

[45]

Conformiq. http://www.conformiq.com.

[46]

C. Csallner and Y. Smaragdakis. JCrasher: An automatic robustness tester for Java. Software Practice and Experience, 34(11):1025–1050, Sept. 2004.

Digital Library

[47]

C. Csallner and Y. Smaragdakis. Check’n’crash: combining static checking and testing. In Proceedings of the 27th International Conference on Software Engineering, pages 422–431. ACM, 2005.

Digital Library

[48]

C. Csallner, Y. Smaragdakis, and T. Xie. Dsd-crasher: A hybrid analysis tool for bug finding. ACM Transactions on Software Engineering and Methodology (TOSEM), 17(2):8:1–8:37, 2008.

Digital Library

[49]

J. Czerwonka. Pairwise testing in the real world: Practical extensions to test-case scenarios. In Proceedings of the Pacific Northwest Sofware Quality Conference, 2006.

[50]

V. Dallmeier and T. Zimmermann. Extraction of bug localization benchmarks from history. In Proceedings of Automated Software Engineering, pages 433–436, Nov. 2007.

Digital Library

[51]

J. A. De Loera, R. Hemmecke, J. Tauzer, and R. Yoshida. Effective lattice point counting in rational convex polytopes. Journal of Symbolic Computation, 38(4):1273–1302, 2004.

[52]

L. De Moura and N. Bjørner. Z3: An efficient SMT solver. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 337–340, 2008.

Digital Library

[53]

G. Demiroz and C. Yilmaz. Cost-aware combinatorial interaction testing. In Proceedings of the Internatinoal Conference on Advances in System Testing and Validation Lifecycles, pages 9–16, Nov. 2012.

[54]

W. Dickinson, D. Leon, and A. Podgurski. Finding failures by cluster analysis of execution profiles. In Proceedings of the International Conference on Software Engineering, pages 339–348, 2001.

Digital Library

[55]

W. Dietl, S. Dietzel, M. D. Ernst, N. Mote, B. Walker, S. Cooper, T. Pavlik, and Z. Popovi´c. Verification games: Making verification fun. In Proceedings of the Workshop on Formal Techniques for Java-like Programs, pages 42–49, 2012.

Digital Library

[56]

H. Do, S. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal, 10(4):405–435, 2005.

Digital Library

[57]

B. Dutertre and L. de Moura. The YICES SMT Solver.

[58]

P. M. Duvall, S. Matyas, and A. Glover. Continuous Integration: Improving Software Quality and Reducing Risk. Pearson Education, 2007.

Digital Library

[59]

Eclipse Foundation. Eclipse. http://www.eclipse.org/, 2014.

[60]

S. Elbaum and M. Hardojo. An empirical study of profiling strategies for released software and their impact on testing activities. In Proceedings of the International Symposium on Software Testing and Analysis, pages 65–75, 2004.

Digital Library

[61]

E. Engström and P. Runeson. A qualitative survey of regression testing practices. In Proceedings of the International Conference on Product-Focused Software Process Improvement, pages 3–16, 2010.

Digital Library

[62]

M. D. Ernst, J. H. Perkins, P. J. Guo, S. McCamant, C. Pacheco, M. S. Tschantz, and C. Xiao. The Daikon system for dynamic detection of likely invariants. Science of Computer Programming, 69(1):35–45, 2007.

Digital Library

[63]

A. Filieri, C. S. Păsăreanu, and W. Visser. Reliability analysis in symbolic pathfinder. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 622–631, Piscataway, NJ, USA, 2013. IEEE Press.

Digital Library

[64]

M. Fowler. Continuous integration. martinflowler.com/articles/continuousIntegration.html.

[65]

G. Fraser, M. Staats, P. McMinn, A. Arcuri, and F. Padberg. Does automated white-box test generation really help software testers? In Proceedings of the International Symposium on Software Testing and Analysis, pages 188–198, July 2013.

Digital Library

[66]

G. Fraser and A. Zeller. Mutation-driven generation of unit tests and oracles. IEEE Transactions on Software Engineering, 38(2):278–292, 2012.

Digital Library

[67]

J. P. Galeotti, G. Fraser, and A. Arcuri. Improving search-based test suite generation with dynamic symbolic execution. In Proceedings of the International Symposium on Softaware Reliability Engineering, pages 360–369, 2013.

[68]

D. Garg and A. Datta. Test case prioritization due to database changes in web applications. In Proceedings of the International Conference on Software Testing, pages 726–730, Apr. 2012.

Digital Library

[69]

B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering, 16(1):61–102, Feb. 2011.

Digital Library

[70]

J. Geldenhuys, M. B. Dwyer, and W. Visser. Probabilistic symbolic execution. In Proceedings of the International Symposium on Software Testing and Analysis, pages 166–176, 2012.

Digital Library

[71]

K. Glerum, K. Kinshumann, S. Greenberg, G. Aul, V. Orgovan, G. Nichols, D. Grant, G. Loihle, and G. Hunt. Debugging in the (very) large: Ten years of implementation and experience. In Proceedings of the ACM SIGOPS Symposium on Operating Systems Principles, pages 103–116, 2009.

Digital Library

[72]

P. Godefroid. Higher-order test generation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 258–269, 2011.

Digital Library

[73]

P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 213–223, 2005.

Digital Library

[74]

P. Godefroid, M. Y. Levin, and D. Molnar. Sage: Whitebox fuzzing for security testing. Queue, 10(1):20:20–20:27, Jan. 2012.

Digital Library

[75]

W. Grieskamp, N. Kicillof, K. Stobie, and V. Braberman. Model-based quality assurance of protocol documentation: Tools and methodology. Journal of Software Testing, Verification, and Reliability, 21:55–71, 2010.

Digital Library

[76]

W. Grieskamp, X. Qu, X. Wei, N. Kicillof, and M. B. Cohen. Interaction coverage meets path coverage by smt constraint solving. In Joint Conference of the IFIP International Conference on Testing of Communicating Systems and International Workshop on Formal Approaches to Testing of Software, 2009.

Digital Library

[77]

M. Grindal and J. Offutt. Input parameter modeling for combination strategies. In Proceedings of the Conference on IASTED International Multi-Conference: Software Engineering, pages 255–260, 2007.

Digital Library

[78]

A. Groce, C. Zhang, E. Eide, Y. Chen, and J. Regehr. Swarm testing. In Proceedings of the International Symposium on Software Testing and Analysis, pages 78–88, 2012.

Digital Library

[79]

F. Gross, G. Fraser, and A. Zeller. Search-based system testing: High coverage, no false alarms. In Proceedings of the International Symposium on Software Testing and Analysis, pages 67–77, 2012.

Digital Library

[80]

B. S. Gulavani, T. A. Henzinger, Y. Kannan, A. V. Nori, and S. K. Rajamani. Synergy: A new algorithm for property checking. In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 117–127, 2006.

Digital Library

[81]

N. Gupta, A. P. Mathur, and M. L. Soffa. Generating test data for branch coverage. In Proceedings of the IEEE International Conference on Automated Software Engineering, pages 219–227, September 2000.

Digital Library

[82]

P. Gupta, M. Ivey, and J. Penix. Testing at the speed and scale of google. http://googletesting.blogspot.com/2011/ 06/testing-at-speed-and-scale-of-google.html, June 2011.

[83]

S. Hao, D. Li, W. G. J. Halfond, and R. Govindan. Estimating mobile application energy consumption using program analysis. In Proceedings of the 2013 International Conference on Software Engineering (ICSE 2013), pages 92–101, 2013.

Digital Library

[84]

M. Harman, L. Hu, R. Hierons, J. Wegener, H. Sthamer, A. Baresel, and M. Roper. Testability transformation. IEEE Transactions on Software Engineering, 30(1):3–16, Jan. 2004.

Digital Library

[85]

M. Harman, A. Mansouri, and Y. Zhang. Search based software engineering: A comprehensive analysis and review of trends, techniques and applications. Technical Report Technical Report TR-09-03, King’s College London, 2009.

[86]

M. Harman, S. A. Mansouri, and Y. Zhang. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys, 45(1):11:1–11:61, Dec. 2012.

Digital Library

[87]

M. Harman, P. McMinn, M. Shahbaz, and S. Yoo. A comprehensive survey of trends in oracles for software testing. Technical report, Research Memoranda CS-13-01, Department of Computer Science, University of Sheffield, 2013.

[88]

M. J. Harrold. Testing: A roadmap. In Proceedings of the Conference on the Future of Software Engineering, pages 61–72, 2000.

Digital Library

[89]

M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Transactions on Software Engineering and Methodology, 2(3):270–285, July 1993.

Digital Library

[90]

A. Hartman and R. L. Problems and algorithms for covering arrays. Discrete Mathematics, 248:149–156, 2004.

Digital Library

[91]

H. Hemmati, A. Arcuri, and L. Briand. Achieving scalable model-based testing through test case diversity. ACM Transactions on Software Engineering and Methodology, 22(1):6:1–6:42, 2012.

Digital Library

[92]

A. Hinton, M. Kwiatkowska, G. Norman, and D. Parker. Prism: A tool for automatic verification of probabilistic systems. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 441–444, 2006.

Digital Library

[93]

A. Huima. Implementing Conformiq Qtronic. In A. Petrenko, M. Veanes, J. Tretmans, and W. Grieskamp, editors, Testing of Software and Communicating Systems, volume 4581 of Lecture Notes in Computer Science, pages 1–12. 2007.

Digital Library

[94]

M. Hutchins, H. Foster, T. Goradia, and T. Ostrand. Experiments on the effectiveness of dataflow- and controlflow-based test adequacy criteria. In Proceedings of the International Conference on Software Engineering, pages 191–200, May 1994.

Digital Library

[95]

K. Inkumsah and T. Xie. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proceedings of the International Conference on Automated Softwaare Engineering, pages 297–306, 2008.

Digital Library

[96]

Jenkins. Jenkins: An extendable open source continous integration server. jenkins-ci.org.

[97]

W. Jin and A. Orso. BugRedux: Reproducing field failures for in-house debugging. In Proceedings of the International Conference on Software Engineering, pages 474–484, 2012.

Digital Library

[98]

Apache JMeter. https://jmeter.apache.org.

[99]

J. A. Jones, A. Orso, and M. J. Harrold. Gammatella: Visualizing program-execution data for deployed software. Information Visualization, 3(3):173–188, 2004.

Digital Library

[100]

JUnit Testing Framework. http://www.junit.org, 2014.

[101]

E. Y. Kan. Energy efficiency in testing and regression testing: a comparison of DVFS techniques. In Proceedings of the International Conference on Quality Software, pages 280–283, 2013.

Digital Library

[102]

S. Khurshid, C. S. Păsăreanu, and W. Visser. Generalized symbolic execution for model checking and testing. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 553–568, 2003.

Digital Library

[103]

C. H. P. Kim, D. S. Batory, and S. Khurshid. Reducing combinatorics in testing product lines. In Proceedings of the International Conference on Aspect-Oriented Software Development, pages 57–68, 2011.

Digital Library

[104]

J. Kim, H. Oh, H. Ha, S.-H. Kang, J. Choi, and S. Ha. An ILP-based worst-case performance analysis technique for distributed real-time embedded systems. In Proceedings of the Real-time Systems Symposium, pages 363–372, 2012.

Digital Library

[105]

M. Kim, Y. Kim, and G. Rothermel. A scalable distributed concolic testing approach: An empirical evaluation. In Proceedings of the International Conference on Software Testing, pages 340–349, Apr. 2012.

Digital Library

[106]

Y. Kim, Z. Xu, M. Kim, M. B. Cohen, and G. Rothermel. Hybrid directed test suite augmentation: An interleaving framework. In Proceedings of the International Conference on Software Testing, Apr. 2014.

Digital Library

[107]

J. C. King. Symbolic Execution and Program Testing. Communications of the ACM, 19(7):385–394, 1976.

Digital Library

[108]

R. Kirner, P. Puschner, and I. Wenzel. Measurement-based worst-case execution time analysis using automatic test-data generation. In Proceedings of the IEEE Workshop on Software Technology for Future Embedded and Ubiquitous Systems, pages 7–10, 2004.

Digital Library

[109]

B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. E. Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8):721–734, Aug. 2002.

Digital Library

[110]

B. Korel. A Dynamic Approach of Test Data Generation. In Proceedings of the IEEE Conference on Software Maintenance, pages 311–317, November 1990.

[111]

R. Krishnan, S. M. Krishna, and P. S. Nandhan. Combinatorial testing: Learnings from our experience. SIGSOFT Software Engineering Notes, 32(3):1–8, May 2007.

Digital Library

[112]

LabView System Design Software. http://www.ni.com/labview/.

[113]

B. Legeard and M. Utting. Model-based testing - next generation functional testing. SoftwareTech News, 12(4):9–18, Jan. 2010.

[114]

D. Li, S. Hao, W. G. J. Halfond, and R. Govindan. Calculating source line level energy information for android applications. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, pages 78–89, 2013.

Digital Library

[115]

D. Li, Y. Jin, C. Sahin, J. Clause, and W. G. J. Halfond. Integrated energy-directed test suite optimization. In Proceedings of the International Symposium on Software Testing and Analysis, July 2014.

Digital Library

[116]

B. Liblit, M. Naik, A. X. Zheng, A. Aiken, and M. I. Jordan. Scalable statistical bug isolation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 15–26, 2005.

Digital Library

[117]

Y. Lin, X. Tang, Y. Chen, and J. Zhao. A divergence-oriented approach to adaptive random testing of Java programs. In Proceedings of the International Conference on Automated Software Engineering, pages 221–232, 2009.

Digital Library

[118]

Y. Liu, C. Xu, and S. C. Cheung. Where has my battery gone? Finding sensor related energy black holes in smartphone applications. In Proceedings of the International Conference on Pervasive Computing and Communications, pages 2–10, 2013.

[119]

C. Lott, A. Jain, and S. Dalal. Modeling requirements for combinatorial software testing. In Proceedings of the 1st International Workshop on Advances in Model-based Testing, pages 1–7, 2005.

Digital Library

[120]

S. Lu, Z. Li, F. Qin, L. Tan, P. Zhou, and Y. Zhou. Bugbench: Benchmarks for evaluating bug detection tools. In Proceedings of the Workshop on the Evaluation of Software Defect Detection Tools, 2005.

[121]

A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for android apps. In Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 224–234, 2013.

Digital Library

[122]

J. Malburg and G. Fraser. Combining search-based and constraint-based testing. In Proceedings of the International Conference on Automated Softwaare Engineering, pages 436–439, 2011.

Digital Library

[123]

A. Marchetto, P. Tonella, and F. Ricca. State-based testing of ajax web applications. In Proceedings of the International Conference on Software Testing, pages 121–130, 2008.

Digital Library

[124]

L. Mariani, M. Pezzè, O. Riganelli, and M. Santoro. AutoBlackTest: A tool for automatic black-box testing. In Proceedings of the International Conference on Software Engineering, pages 1013–1015, 2011.

Digital Library

[125]

Matlab. http://www.mathworks.com/products/matlab/.

[126]

S. McMaster and A. Memon. Call-stack covereage for GUI test suite reduction. IEEE Transactions on Software Engineering, 34(1):99–115, 2008.

Digital Library

[127]

P. McMinn. Search-based software test data generation: A survey. Journal of Software Testing, Verification, and Reliability, 14(2):105–156, June 2004.

Digital Library

[128]

P. McMinn. Search-based software testing: past, present, and future. In Proceedings of the International Workshop on Search-Based Software Testing, pages 153–163, Mar. 2011.

Digital Library

[129]

P. McMinn, D. Binkley, and M. Harman. Empirical evaluation of a nesting testability transformation for evolutionary testing. ACM Transactions on Software Engineering and Methodology, 18(3):11:1–11:27, June 2009.

Digital Library

[130]

A. Memon, I. Banerjee, and A. Nagarajan. What test oracle should I use for effective GUI testing? In Proceedings of Automated Software Engineering, pages 164–173, Nov. 2003.

Digital Library

[131]

D. Monniaux. An abstract monte-carlo method for the analysis of probabilistic programs. In Proceedings of the ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 93–101, 2001.

Digital Library

[132]

A. Morgado, F. Heras, and J. Marques-Silva. Improvements to Core-guided Binary Search for MaxSAT. In Proceedings of the International Conference on Theory and Applications of Satisfiability Testing, pages 284–297, 2012.

Digital Library

[133]

Mozilla Developer Network. Tinderbox. https: //developer.mozilla.org/en-US/docs/Tinderbox, 2014.

[134]

National Institute of Standards and Technology. Nist software assurance reference dataset project. samate.nist.gov/SRD/.

[135]

Nester. nester.sourceforge.net.

[136]

C. Nie and H. Leung. A survey of combinatorial testing. ACM Computing Surveys, 43(2):11:1–11:29, Feb. 2011.

Digital Library

[137]

K. Nurmela. Upper bounds for covering arrays by tabu search. Discrete Applied Mathematics, 138(1-2):143–152, 2004.

Digital Library

[138]

J. Offut, Y.-S. Ma, and Y.-R. Kown. MuJava: An automated class mutation system. Journal of Software Testing, Verification, and Reliability, 15(2):97–133, June 2005.

Digital Library

[139]

A. J. Offutt, Z. Jin, and J. Pan. The dynamic domain reduction procedure for test data generation. Software Practice and Experience, 29(2):167–193, 1997.

Digital Library

[140]

Open Source Software Testing Tools. www.opensourcetesting.org/performance.php.

[141]

A. Orso, N. Shi, and M. J. Harrold. Scaling regression testing to large software systems. In Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 241–252, Nov. 2004.

Digital Library

[142]

S. Oster, M. Zink, M. Lochau, and M. Grechanik. Pairwise feature-interaction testing for SPLs: potentials and limitations. In Proceedings of the International Conference on Software Product Lines, pages 6:1–6:8, 2011.

Digital Library

[143]

L. Osterweil. Strategic directions in software quality. ACM Computing Surveys, 4:738–750, Dec. 1996.

Digital Library

[144]

C. Pacheco and M. Ernst. Eclat: Automatic generation and classification of test inputs. Proceedings of the European Conference on Object-Oriented Programming, pages 504–527, 2005.

Digital Library

[145]

C. Pacheco and M. D. Ernst. Randoop: Feedback-directed random Testing for Java. In Proceedings of the ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications, pages 815–816, 2007.

Digital Library

[146]

C. Pacheco, S. K. Lahiri, and T. Ball. Finding Errors in .NET with Feedback-Directed Random Testing. In Proceedings of the 2008 International Symposium on Software Testing and Analysis, pages 87–96, 2008.

Digital Library

[147]

C. Parnin and A. Orso. Are automated debugging techniques actually helping programmers? In Proceedings of the International Symposium on Software Testing and Analysis, pages 199–209, July 2011.

Digital Library

[148]

F. Pastore, L. Mariani, and G. Fraser. CrowdOracles: Can the crowd solve the oracle problem? In Proceedings of the International Conference on Software Testing, Apr. 2013.

Digital Library

[149]

Performance Testing Guidance for Web Applications. perftestingguide.codeplex.com/releases/view/.

[150]

C. S. Păsăreanu, N. Rungta, and W. Visser. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of the 2011 International Symposium on Software Testing and Analysis, pages 34–44, 2011.

Digital Library

[151]

P. Puschner and R. Nossal. Testing the results of static worst-case execution-time analysis. In Proceedings of the real-time systems symposium, pages 134–143, Dec. 1998.

Digital Library

[152]

The r project for statistical computing. http://www.r-project.org/.

[153]

M. Research. The yogi project, 2014. http://research.microsoft.com/en-us/projects/Yogi.

[154]

G. Rothermel, M. Burnett, L. Li, C. Dupuis, and A. Sheretov. A methodology for testing spreadsheets. ACM Transactions on Software Engineering and Methodology, 10(1):110–147, Jan. 2001.

Digital Library

[155]

G. Rothermel and M. J. Harrold. Selecting tests and identifying test coverage requirements for modified software. In Proceedings of the International Symposium on Software Testing and Analysis, pages 169–184, Aug. 1994.

Digital Library

[156]

G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering, 22(8):529–551, Aug. 1996.

Digital Library

[157]

G. Rothermel and M. J. Harrold. A safe, efficient regression test selection technique. ACM Transactions on Software Engineering and Methodology, 6(2):173–210, Apr. 1997.

Digital Library

[158]

G. Rothermel, M. J. Harrold, and J. Dedhia. Regression test selection for C++ programs. Journal of Software Testing, Verification, and Reliability, 10(2):77–109, June 2000.

[159]

G. Rothermel, R. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27(10):929–948, Oct. 2001.

Digital Library

[160]

P. Runeson. A survey of unit testing practices. IEEE Softw., 23(4):22–29, July 2006.

Digital Library

[161]

P. Runeson, M. Host, A. Rainer, and B. Regnell. Case Study Research in Software Engineering. John Wiley and Sons, Hoboken, NJ, 2012.

Digital Library

[162]

S. Sampath, R. C. Bryce, G. Viswanath, and V. Kandimalla. Prioritizing user-session-based test cases for web application testing. In Proceedings of the International Conference on Software Testing, pages 141–150, Apr. 2008.

Digital Library

[163]

S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis for probabilistic programs: Inferring whole program properties from finitely many paths. SIGPLAN Notices, 48(6):447–458, June 2013.

Digital Library

[164]

R. Santelices, P. K. Chittimalli, T. Apiwattanapong, A. Orso, and M. J. Harrold. Test-suite augmentation for evolving software. In Proceedings of Automated Software Engineering, Sept. 2008.

Digital Library

[165]

SAT4J. http://www.sat4j.org, 2012.

[166]

D. Schuler and A. Zeller. Javalanche: Efficient mutation testing for Java. In Proceedings of the ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 297–298, 2009.

Digital Library

[167]

K. Schwaber and M. Beedle. Agile Software Development with Scrum. Prentice Hall PTR, Upper Saddle River, NJ, USA, 1st edition, 2001.

Digital Library

[168]

I. Segall, R. Tzoref-Brill, and E. Farchi. Using binary decision diagrams for combinatorial test design. In Proceedings of the International Symposium on Software Testing and Analysis, pages 254–264, 2011.

Digital Library

[169]

D. Sehlberg, A. Ermedahl, J. Gustafsson, B. Lisper, and S. Wiegratz. Static WCET analysis of real-time task-oriented code in vehicle control systems. In Proceedings of the Second International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, pages 212–219, 2006.

Digital Library

[170]

K. Sen. Effective random testing of concurrent programs. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, pages 323–332, 2007.

Digital Library

[171]

K. Sen. Race directed random testing of concurrent programs. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 11–21, 2008.

Digital Library

[172]

K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 263–272, 2005.

Digital Library

[173]

A. Shahbazi, A. F. Tappenden, and J. Miller. Centroidal Voronoi tessellations — A new approach to random testing. IEEE Transactions on Software Engineering, 39(2):163–183, 2013.

Digital Library

[174]

G. B. Sherwood, S. S. Martirosyan, and C. J. Colbourn. Covering arrays of higher strength from permutation vectors. Journal of Combinatorial Design, 3(14):202–213, 2005.

[175]

T. Shiba, T. Tsuchiya, and T. Kikuno. Using artificial life techniques to generate test cases for combinatorial testing. In Proceedings of the Annual International Computer Software and Applications Conference, pages 72–77, 2004.

Digital Library

[176]

Software-artifact Infrastructure Repository. http://sir.unl.edu/, Apr. 2012.

[177]

Smartesting. http://www.smartesting.com.

[178]

D. Song, D. Brumley, H. Yin, J. Caballero, I. Jager, M. G. Kang, Z. Liang, J. Newsome, P. Poosankam, and P. Saxena. Bitblaze: A new approach to computer security via binary analysis. In Proceedings of the International Conference on Information Systems Security, pages 1–25, 2008.

Digital Library

[179]

J. Spacco, J. Strecker, D. Hovemeyer, and W. Pugh. Software repository mining with marmoset: An automated programming project snapshot and testing system. In Proceedings of the International Workshop on Mining Software Repositories, pages 1–5, 2005.

Digital Library

[180]

Spec explorer. http://www.specexplorer.net.

[181]

A. Srivastava and J. Thiagarajan. Effectively prioritizing tests in development environment. In Proceedings of the International Symposium on Software Testing and Analysis, 2002.

Digital Library

[182]

M. Staats, G. Gay, and M. Heimdahl. Automated oracle creation support, or: How I learned to stop worrying about fault propagation and love mutation testing. In Proceedings of the International Conference on Software Engineering, pages 870–880, May 2012.

Digital Library

[183]

M. Staats, S. Hong, M. Kim, and G. Rothermel. Understanding user understanding: Determining correctness of generated program invariants. In Proceedings of the International Symposium on Software Testing and Analysis, pages 188–198, July 2012.

Digital Library

[184]

M. Staats and C. Pˇasˇareanu. Parallel symbolic execution for structural test generation. In Proceedings of the International Symposium on Software Testing and Analysis, pages 183–194, July 2010.

Digital Library

[185]

M. Staats, M. W. Whalen, and M. Heimdahl. Programs, tests, and oracles: The foundations of testing revisited. In Proceedings of the International Conference on Software Engineering, pages 391–400, 2011.

Digital Library

[186]

K. Taneja and T. Xie. Diffgen: Automated regression unit-test generation. In Proceedings of Automated Software Engineering, pages 407–410, Nov. 2008.

Digital Library

[187]

K. Taneja, T. Xie, N. Tillmann, J. Halleux, and W. Schulte. eXpress: Guided path exploration for regression test generation. In Proceedings of the International Symposium on Software Testing and Analysis, July 2011.

Digital Library

[188]

A. F. Tappenden and J. Miller. A novel evolutionary approach for adaptive random testing. IEEE Transactions on Reliability, 58(4):619–633, 2009.

[189]

Test Studio. www.telerik.com/teststudio/performance-testing.

[190]

ThoughtWorks. Go: Continous delivery. www.thoughtworks.com/products/go-continuous-delivery.

[191]

N. Tillmann and J. De Halleux. Pex: White box test generation for .net. In Proceedings of the International Conference on Tests and Proofs, pages 134–153, 2008.

Digital Library

[192]

TransWorldNews. Amazon.com Down for Hours After Unknown Outage. http://www.transworldnews.com/ NewsStory.aspx?storyid=49790, 2010.

[193]

M. Utting, A. Pretschner, and B. Legeard. A taxonomy of model-based testing approaches. Journal of Software Testing, Verification, and Reliability, 22(5):297–312, 2012.

Digital Library

[194]

E. Uzuncaova, D. Garcia, S. Khurshid, and D. Batory. Testing software product lines using incremental test generation. In Proceedings of the International Symposium on Software Reliability Engineering, pages 249–258, 2008.

Digital Library

[195]

Y. Wei, C. A. Furia, N. Kazmin, and B. Meyer. Inferring better contracts. In Proceedings of the International Conference on Software Engineering, pages 191–200, May 2011.

Digital Library

[196]

E. J. Weyuker. On testing non-testable programs. The Computer Journal, 15(4):465–470, 1982.

[197]

L. White and B. Robinson. Industrial real-time regression testing and analysis using firewalls. In Proceedings of the International Conference on Software Maintenance, Sept. 2004.

Digital Library

[198]

C. Wohlin, P. Runeson, M. Host, M. Ohlsson, B. Regnell, and A. Wesslen. Experimentation in Software Engineering. Kluwer Academic Publishers, Norwell, MA, 2000.

[199]

Q. Xie and A. M. Memon. Designing and comparing automated test oracles for GUI-based software applications. ACM Transactions on Software Engineering and Methodology, 16(1):4, 2007.

Digital Library

[200]

T. Xie, N. Tillmann, J. de Halleux, and W. Schulte. Fitness-guided Path Exploration in Dynamic Symbolic Execution. In Proceedings of the International Conference on Dependable Systems and Networks, pages 359–368, 2009.

[201]

Z. Xu, Y. Kim, M. Kim, G. Rothermel, and M. Cohen. Directed test suite augmentation: Techniques and tradeoffs. In Proceedings of the ACM SIGSOFT Symposium on Foundations of Software Engineering, Nov. 2010.

Digital Library

[202]

Z. Xu, Y. Kim, K. M, and G. Rothermel. A hybrid directed test suite augmentation technique. In Proceedings of the International Symposium on Software Reliability Engineering, 2011.

Digital Library

[203]

Yahoo! pipes. http://pipes.yahoo.com/pipes/.

[204]

C. Yilmaz. Test case-aware combinatorial interaction testing. IEEE Transactions on Software Engineering, 39(5):684–706, May 2013.

Digital Library

[205]

C. Yilmaz, S. Fouche, M. B. Cohen, A. Porter, G. Demiroz, and U. Koc. Moving forward with combinatorial interaction testing. IEEE Computer, 47(2):37–45, Feb. 2014.

Digital Library

[206]

S. Yoo and M. Harman. Regression testing minimisation, selection and prioritisation: A survey. Software Testing, Verification and Reliability, 22(2), 2012.

Digital Library

[207]

S. Yoo, M. Harman, P. Tonella, and A. Susi. Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge. In Proceedings of the International Symposium on Software Testing and Analysis, 2009.

Digital Library

[208]

T. Yu, W. Srisa-an, and G. Rothermel. SimLatte: A framework to support testing for worst-case interrupt latencies in embedded software. In Proceedings of the International Conference on Software Testing, Apr. 2014.

Digital Library

[209]

T. Yu, W. Srisa-an, and G. Rothermel. SimRT: An automated framework to support regression testing for data races. In Proceedings of the International Conference on Software Engineering, June 2014.

Digital Library

[210]

C. Zamfir and G. Candea. Execution Synthesis: A Technique for Automated Software Debugging. In Proceedings of the 5th European Conference on Computer Systems, pages 321–334, 2010.

Digital Library

Cited By

Stef MPolgar Z(2024)Software Platform for the Comprehensive Testing of Transmission Protocols Developed in GNU RadioInformation10.3390/info1501006215:1(62)Online publication date: 20-Jan-2024
https://doi.org/10.3390/info15010062
Gupt KKshirsagar MDias DSullivan JRyan C(2024)A novel ML-driven test case selection approach for enhancing the performance of grammatical evolutionFrontiers in Computer Science10.3389/fcomp.2024.13461496Online publication date: 27-Jun-2024
https://doi.org/10.3389/fcomp.2024.1346149
Cohen Md'Amorim M(2024)It’s Organic: Software Testing of Emerging Domains (Keynote)Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3674720(2-3)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3674720
Show More Cited By

Index Terms

Software testing: a research travelogue (2000–2014)

Index terms have been assigned to the content through auto-classification.

Recommendations

Testing research software: a survey
AbstractBackground
Research software plays an important role in solving real-life problems, empowering scientific innovations, and handling emergency situations. Therefore, the correctness and trustworthiness of research software are of absolute ...
Testing scientific software: A systematic literature review

Context: Scientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract ...
Global software testing under deadline pressure: Vendor-side experiences

Context: In the era of globally-distributed software engineering, the practice of global software testing (GST) has witnessed increasing adoption. Although there have been ethnographic studies of the development aspects of global software engineering, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FOSE 2014: Future of Software Engineering Proceedings

May 2014

224 pages

ISBN:9781450328654

DOI:10.1145/2593882

General Chair:
James Herbsleb
Carnegie Mellon University, USA
,
Program Chairs:
Matthew B. Dwyer
University of Nebraska at Lincoln, USA
,
James Herbsleb
Carnegie Mellon University, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

Software testing

Qualifiers

Article

Conference

ICSE '14

Sponsor:

SIGSOFT

ICSE '14: 36th International Conference on Software Engineering

May 31 - June 7, 2014

Hyderabad, India

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

150
Total Citations
View Citations
3,458
Total Downloads

Downloads (Last 12 months)217
Downloads (Last 6 weeks)15

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stef MPolgar Z(2024)Software Platform for the Comprehensive Testing of Transmission Protocols Developed in GNU RadioInformation10.3390/info1501006215:1(62)Online publication date: 20-Jan-2024
https://doi.org/10.3390/info15010062
Gupt KKshirsagar MDias DSullivan JRyan C(2024)A novel ML-driven test case selection approach for enhancing the performance of grammatical evolutionFrontiers in Computer Science10.3389/fcomp.2024.13461496Online publication date: 27-Jun-2024
https://doi.org/10.3389/fcomp.2024.1346149
Cohen Md'Amorim M(2024)It’s Organic: Software Testing of Emerging Domains (Keynote)Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3674720(2-3)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3674720
Huang RCui CLian JTowey DSun WChen H(2024)Toward Cost-Effective Adaptive Random Testing: An Approximate Nearest Neighbor ApproachIEEE Transactions on Software Engineering10.1109/TSE.2024.337959250:5(1182-1214)Online publication date: 21-Mar-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3379592
Metwalli SVan Meter R(2024)Testing and Debugging Quantum CircuitsIEEE Transactions on Quantum Engineering10.1109/TQE.2024.33748795(1-15)Online publication date: 2024
https://doi.org/10.1109/TQE.2024.3374879
Agrawal AZech PVierhauser M(2024)Coupled Requirements-Driven Testing of CPS: From Simulation to Reality2024 IEEE 32nd International Requirements Engineering Conference (RE)10.1109/RE59067.2024.00040(337-344)Online publication date: 24-Jun-2024
https://doi.org/10.1109/RE59067.2024.00040
Khan SWang WRen YZhu BShi JMcGowan ALam WMoran K(2024)Aurora: Navigating UI Tarpits via Automated Neural Screen Understanding2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00028(221-232)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00028
Retzlaff N(2024)AI Integrated ST Modern System for Designing Automated Standard Affirmation System2024 4th International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE)10.1109/ICACITE60783.2024.10616416(1386-1391)Online publication date: 14-May-2024
https://doi.org/10.1109/ICACITE60783.2024.10616416
Sebastian ANaseem HCatal C(2024)Unsupervised Machine Learning Approaches for Test Suite ReductionApplied Artificial Intelligence10.1080/08839514.2024.232233638:1Online publication date: 4-Mar-2024
https://doi.org/10.1080/08839514.2024.2322336
Ying ZTowey DBellotti AChen TZhou Z(2024)SFIDMT-ART: A metamorphic group generation method based on Adaptive Random Testing applied to source and follow-up input domainsInformation and Software Technology10.1016/j.infsof.2024.107528(107528)Online publication date: Jul-2024
https://doi.org/10.1016/j.infsof.2024.107528
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten