skip to main content
survey

A Survey on Data-Flow Testing

Published: 10 March 2017 Publication History

Abstract

Data-flow testing (DFT) is a family of testing strategies designed to verify the interactions between each program variable’s definition and its uses. Such a test objective of interest is referred to as a def-use pair. DFT selects test data with respect to various test adequacy criteria (i.e., data-flow coverage criteria) to exercise each pair. The original conception of DFT was introduced by Herman in 1976. Since then, a number of studies have been conducted, both theoretically and empirically, to analyze DFT’s complexity and effectiveness. In the past four decades, DFT has been continuously concerned, and various approaches from different aspects are proposed to pursue automatic and efficient data-flow testing. This survey presents a detailed overview of data-flow testing, including challenges and approaches in enforcing and automating it: (1) it introduces the data-flow analysis techniques that are used to identify def-use pairs; (2) it classifies and discusses techniques for data-flow-based test data generation, such as search-based testing, random testing, collateral-coverage-based testing, symbolic-execution-based testing, and model-checking-based testing; (3) it discusses techniques for tracking data-flow coverage; (4) it presents several DFT applications, including software fault localization, web security testing, and specification consistency checking; and (5) it summarizes recent advances and discusses future research directions toward more practical data-flow testing.

References

[1]
Hiralal Agrawal, Joseph R. Horgan, Saul London, and W. Eric Wong. 1995. Fault localization using execution slices and dataflow tests. In Proceedings of the Sixth International Symposium on Software Reliability Engineering (ISSRE’95). 143--151.
[2]
Alfred V. Aho, Ravi Sethi, and Jeffrey D. Ullman. 1986. Compilers: Principles, Techniques, and Tools. Addison-Wesley Longman Publishing Co., Boston, MA.
[3]
Roger T. Alexander, Jeff Offutt, and Andreas Stefik. 2010. Testing coupling relationships in object-oriented programs. Softw. Test., Verif. Reliab. 20, 4 (2010), 291--327.
[4]
Frances E. Allen and John Cocke. 1976. A program data flow analysis procedure. Commun. ACM 19, 3 (1976), 137--147.
[5]
Nadia Alshahwan and Mark Harman. 2012. State aware test case regeneration for improving web application test suite coverage and fault detection. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’12). 45--55.
[6]
Paul Ammann, A. Jefferson Offutt, and Hong Huang. 2003. Coverage criteria for logical expressions. In Proceedings of the International Symposium on Software Reliability Engineering. 99--107.
[7]
Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing. Cambridge University Press, New York, NY.
[8]
Saswat Anand, Edmund K. Burke, Tsong Yueh Chen, John A. Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, and Phil McMinn. 2013. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software 86, 8 (2013), 1978--2001.
[9]
Andrea Arcuri and Lionel C. Briand. 2011. Adaptive random testing: An illusion of effectiveness? In Proceedings of the 20th International Symposium on Software Testing and Analysis (ISSTA’11). 265--275.
[10]
Zeina Awedikian, Kamel Ayari, and Giuliano Antoniol. 2009. MC/DC automatic test input data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’09). 1657--1664.
[11]
Arthur I. Baars, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, and Tanja E. J. Vos. 2011. Symbolic search-based testing. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11). 53--62.
[12]
Janvi Badlaney, Rohit Ghatol, and Romit Jadhwani. 2006. An Introduction to Data-Flow Testing. Technical Report NCSU CSC TR-2006-22. Department of Computer Science, North Carolina State University, Raleigh, NC 27695. Retrieved from http://people.eecs.ku.edu/∼aiedian/Teaching/Fa09/814/Lectures/intro-df-testing-1.pdf.
[13]
Roberto Bagnara, Matthieu Carlier, Roberta Gori, and Arnaud Gotlieb. 2013. Symbolic path-oriented test data generation for floating-point programs. In 2013 IEEE 6th International Conference on Software Testing, Verification and Validation. 1--10.
[14]
Thomas Ball and Sriram K. Rajamani. 2002. The SLAM project: Debugging system software via static analysis. In Proceedings of the 29th SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’02). 1--3.
[15]
Mauro Baluda. 2011. Automatic structural testing with abstraction refinement and coarsening. In SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13rd European Software Engineering Conference (ESEC-13). 400--403.
[16]
Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2010. Structural coverage of feasible code. In Proceedings of the 5th Workshop on Automation of Software Test (AST’10). ACM, New York, NY, 59--66.
[17]
Mauro Baluda, Pietro Braione, Giovanni Denaro, and Mauro Pezzè. 2011. Enhancing structural software coverage by incrementally computing branch executability. Softw. Qual. J. 19, 4 (2011), 725--751.
[18]
Luciano Baresi, Pier Luca Lanzi, and Matteo Miraz. 2010. TestFul: An evolutionary test approach for java. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10). 185--194.
[19]
Luciano Baresi and Matteo Miraz. 2010. TestFul: Automatic unit-test generation for Java classes. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 (ICSE’10). 281--284.
[20]
Nels E. Beckman, Aditya V. Nori, Sriram K. Rajamani, and Robert J. Simmons. 2008. Proofs from tests. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’08). 3--14.
[21]
B. Beizer. 1990. Software Testing Techniques. International Thomson Computer Press.
[22]
Dirk Beyer, Adam J. Chlipala, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2004. Generating tests from counterexamples. In Proceedings of the 26th International Conference on Software Engineering (ICSE’04). IEEE Computer Society, 326--335.
[23]
Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2007. The software model checker Blast: Applications to software engineering. Int. J. Softw. Tools Technol. Transf. 9, 5 (Oct. 2007), 505--525.
[24]
Dirk Beyer and M. Erkan Keremoglu. 2011. CPAchecker: A tool for configurable software verification. In CAV. 184--190.
[25]
David L. Bird and Carlos Urias Munoz. 1983. Automatic generation of random self-checking test cases. IBM Syst. J. 22, 3 (1983), 229--245.
[26]
Ilona Bluemke and Artur Rembiszewski. 2009. Dataflow approach to testing java programs. In Proceedings of the 4th International Conference on Dependability of Computer Systems, 2009 (DepCos-RELCOMEX’09). IEEE, 69--76.
[27]
Ilona Bluemke and Artur Rembiszewski. 2012. Dataflow testing of java programs with DFC. Adv. Softw. Eng. Techniques, LNCS 7054, 3 (2012), 215--228.
[28]
Rastislav Bodík, Rajiv Gupta, and Mary Lou Soffa. 1997. Refining data flow information using infeasible paths. SIGSOFT Softw. Eng. Notes 22, 6 (Nov. 1997), 361--377.
[29]
Jacob Burnim and Koushik Sen. 2008. Heuristics for scalable dynamic test generation. In ASE. 443--446.
[30]
Ugo A. Buy, Alessandro Orso, and Mauro Pezzè. 2000. Automated testing of classes. In ISSTA. 39--48.
[31]
Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 209--224.
[32]
Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler. 2006. EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS’06). 322--335.
[33]
Cristian Cadar and Koushik Sen. 2013. Symbolic execution for software testing: Three decades later. Commun. ACM 56, 2 (2013), 82--90.
[34]
M. L. Chaim. 1991. POKE-TOOL-A tool to support structural program testing based on data flow analysis. School of Electrical and Computer Engineering, University of Campinas, Campinas, SP, Brazil.
[35]
Marcos L. Chaim, Anthony Accioly, Delano Medeiros Beder, and Marcelo Morandini. 2011. Evaluating instrumentation strategies by program simulation. IADIS Applied Computing.
[36]
Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013a. An efficient bitwise algorithm for intra-procedural data-flow testing coverage. Inf. Process. Lett. 113, 8 (2013), 293--300.
[37]
Marcos Lordello Chaim and Roberto Paulo Andrioli de Araujo. 2013b. Proof of Correctness of the Bitwise Algorithm for Intra-procedural Data-Flow Testing Coverage. Technical Report PPgSI-001/2013. School of Arts, Sciences and Humanities, University of Sao Paulo. http://ppgsi.each.usp.br/arquivos/RelTec/PPgSI-001_2013.pdf.
[38]
Sagar Chaki, Edmund M. Clarke, Alex Groce, Somesh Jha, and Helmut Veith. 2003. Modular verification of software components in C. In ICSE. 385--395.
[39]
BRamkrishna Chatterjee and Barbara G. Ryder. 1999. Data-Flow-Based Testing of Object-Oriented Libraries. Technical Report DCS-TR-382. Rutgers University.
[40]
T. Y. Chen. 2008. Adaptive random testing. In Proceedings of the 8th International Conference on Quality Software (QSIC’08). 443.
[41]
Ilinca Ciupa, Andreas Leitner, Manuel Oriol, and Bertrand Meyer. 2008. ARTOO: Adaptive random testing for object-oriented software. In Proceedings of the 30th International Conference on Software Engineering (ICSE’08). 71--80.
[42]
Edmund M. Clarke and E. Allen Emerson. 1981. Design and synthesis of synchronization skeletons using branching-time temporal logic. In Logics of Programs, Workshop. 52--71.
[43]
Edmund M. Clarke Jr., Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press, Cambridge, MA.
[44]
Lori A. Clarke, Andy Podgurski, Debra J. Richardson, and Steven J. Zeil. 1989. A formal evaluation of data flow path selection criteria. IEEE Trans. Software Eng. 15, 11 (1989), 1318--1332.
[45]
Lee Copeland. 2003. A Practitioner’s Guide to Software Test Design. Artech House, Norwood, MA.
[46]
Roberto Paulo Andrioli de Araujo and Marcos Lordello Chaim. 2014. Data-flow testing in the large. In Proceedings of the IEEE 7th International Conference on Software Testing, Verification and Validation (ICST’14). 81--90.
[47]
Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2008. Contextual integration testing of classes. In Proceedings of the Fundamental Approaches to Software Engineering (FASE’08), Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS’08). 246--260.
[48]
Giovanni Denaro, Alessandra Gorla, and Mauro Pezzè. 2009. DaTeC: Contextual data flow testing of java classes. In 31st International Conference on Software Engineering (ICSE’09). Companion Volume. 421--422.
[49]
Giovanni Denaro, Alessandro Margara, Mauro Pezzè, and Mattia Vivanti. 2015. Dynamic data flow testing of object oriented systems. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 947--958.
[50]
Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2013. Quantifying the complexity of dataflow testing. In AST. 132--138.
[51]
Giovanni Denaro, Mauro Pezzè, and Mattia Vivanti. 2014. On the right objectives of data flow testing. In ICST. 71--80.
[52]
Mingjie Deng, Rong Chen, and Zhenjun Du. 2009. Automatic test data generation model by combining dataflow analysis with genetic algorithm. In Proceedings of the 2009 Joint Conferences on Pervasive Computing (JCPC’09). 429--434.
[53]
Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1996. A demand-driven analyzer for data flow testing at the integration level. In Proceedings of the 18th International Conference on Software Engineering (ICSE’96). IEEE Computer Society, 575--584.
[54]
Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1997. A practical framework for demand-driven interprocedural data flow analysis. ACM Trans. Program. Lang. Syst. 19, 6 (Nov. 1997), 992--1030.
[55]
Jon Edvardsson. 1999. A Survey on Automatic Test Data Generation. In Proceedings of the 2nd Conference on Computer Science and Engineering. 21--28.
[56]
Lynn M. Foreman and Stuart H. Zweben. 1993. A study of the effectiveness of control and data flow testing strategies. J. Syst. Softw. 21, 3 (1993), 215--228.
[57]
P. G. Frankl. 1987. The Use of Data Flow Information for the Selection and Evaluation of Software Test Data. Ph.D. Dissertation. University of New York, NY.
[58]
Phyllis G. Frankl and Oleg Iakounenko. 1998. Further empirical studies of test effectiveness. In Proceedings of the ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT’98). 153--162.
[59]
P. G. Frankl and S. N. Weiss. 1993. An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans. Softw. Eng. 19, 8 (Aug. 1993), 774--787.
[60]
Phyllis G. Frankl, Stewart N. Weiss, and Elaine J. Weyuker. 1985. Asset: A System to Select and Evaluate Tests. Courant Institute of Mathematical Sciences, New York University.
[61]
Phyllis G. Frankl and Elaine J. Weyuker. 1985. A data flow testing tool. In Proceedings of the 2nd Conference on Software Development Tools, Techniques, and Alternatives. IEEE Computer Society Press, Los Alamitos, CA, 46--53.
[62]
P. G. Frankl and E. J. Weyuker. 1988. An applicable family of data flow testing criteria. IEEE Trans. Softw. Eng. 14, 10 (Oct. 1988), 1483--1498.
[63]
Gordon Fraser and Andrea Arcuri. 2012. Sound empirical evidence in software testing. In Proceedings of the 34th International Conference on Software Engineering (ICSE’12). 178--188.
[64]
Gordon Fraser and Andrea Arcuri. 2013. Whole test suite generation. IEEE Trans. Software Eng. 39, 2 (2013), 276--291.
[65]
Gordon Fraser, Franz Wotawa, and Paul Ammann. 2009. Testing with model checkers: A survey. Softw. Test., Verif. Reliab. 19, 3 (2009), 215--261.
[66]
Kamran Ghani and John A. Clark. 2009. Automatic test data generation for multiple condition and MCDC coverage. In The Proceedings of the 4th International Conference on Software Engineering Advances (ICSEA’09). 152--157.
[67]
Ahmed S. Ghiduk. 2010. A new software data-flow testing approach via ant colony algorithms. Universal J. Comput. Sci. Eng. Technol. 1, 1 (Oct. 2010), 64--72.
[68]
Ahmed S. Ghiduk, Mary Jean Harrold, and Moheb R. Girgis. 2007. Using genetic algorithms to aid test-data generation for data-flow coverage. In APSEC. 41--48.
[69]
Moheb R. Girgis. 1993. Using symbolic execution and data flow criteria to aid test data selection. Softw. Test., Verif. Reliab. 3, 2 (1993), 101--112.
[70]
Moheb R. Girgis. 2005. Automatic test data generation for data flow testing using a genetic algorithm. J. UCS 11, 6 (2005), 898--915.
[71]
Moheb R. Girgis, Ahmed S. Ghiduk, and Eman H. Abd-elkawy. 2014. Automatic generation of data flow test paths using a genetic algorithm. Int. J. Comput. Appl. 89, 12 (March 2014), 29--36.
[72]
Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, New York, NY, 213--223.
[73]
Bhargav S. Gulavani, Thomas A. Henzinger, Yamini Kannan, Aditya V. Nori, and Sriram K. Rajamani. 2006. SYNERGY: A new algorithm for property checking. In SIGSOFT FSE. 117--127.
[74]
Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, open problems and challenges for search based software testing. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation (ICST’15). 1--12.
[75]
Mark Harman, Sung Gon Kim, Kiran Lakhotia, Phil McMinn, and Shin Yoo. 2010. Optimizing for the number of tests generated in search based test data generation with an application to the oracle cost problem. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST’10), Workshop Proceedings. 182--191.
[76]
Mary Jean Harrold. 1994. Performing data flow testing in parallel. In Proceedings of the 8th International Symposium on Parallel Processing. 200--207.
[77]
M. Jean Harrold, Rajiv Gupta, and Mary Lou Soffa. 1993. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2, 3 (July 1993), 270--285.
[78]
Mary Jean Harrold and Brian A Malloy. 1992. Data flow testing of parallelized code. In Proceedings of the Conference on Software Maintenance, 1992. IEEE, 272--281.
[79]
Mary Jean Harrold and Gregg Rothermel. 1994. Performing data flow testing on classes. In SIGSOFT FSE. 154--163.
[80]
Mary Jean Harrold and Mary Lou Soffa. 1994. Efficient computation of interprocedural definition-use chains. ACM Trans. Program. Lang. Syst. 16, 2 (March 1994), 175--204.
[81]
Mohammad Mahdi Hassan and James H. Andrews. 2013. Comparing multi-point stride coverage and dataflow coverage. In Proceedings of the 35th International Conference on Software Engineering (ICSE’13). 172--181.
[82]
Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Grégoire Sutre. 2002. Lazy abstraction. In POPL. 58--70.
[83]
P. M. Herman. 1976. A data flow analysis approach to program testing. Australian Comput. J. 8, 3 (1976), 92--96.
[84]
John H. Holland. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press, Cambridge, MA.
[85]
Hyoung Seok Hong, Sung Deok Cha, Insup Lee, Oleg Sokolsky, and Hasan Ural. 2003. Data flow testing as model checking. In ICSE. 232--243.
[86]
Hyoung Seok Hong, Young Gon Kim, Sung Deok Cha, Doo-Hwan Bae, and Hasan Ural. 2000. A test sequence selection method for statecharts. Softw. Test., Verif. Reliab. 10, 4 (2000), 203--227.
[87]
Hyoung Seok Hong and Hasan Ural. 2005. Dependence testing: Extending data flow testing with control dependence. In Proceedings of the 17th IFIP TC6/WG 6.1 International Conference on Testing of Communicating Systems (TestCom’05). 23--39.
[88]
J. R. Horgan and London. 1992. ATAC: A data flow coverage testing tool for C. In Proceedings of the Symposium on Assessment of Quality Software Development Tools. 2--10.
[89]
J. C. Huang. 1979. Detection of data flow anomaly through program instrumentation. IEEE Trans. Softw. Eng. 5, 3 (1979), 226--236.
[90]
Monica Hutchins, Herbert Foster, Tarak Goradia, and Thomas J. Ostrand. 1994. Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In ICSE. 191--200.
[91]
RTCA. 1992. DO-178b: Software considerations in airborne systems and equipment certification. Requirements and Technical Concepts for Aviation (December 1992).
[92]
Kobi Inkumsah and Tao Xie. 2008. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE’08). 297--306.
[93]
Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14). 435--445.
[94]
Mariam Kamkar, Peter Fritzson, and Nahid Shahmehri. 1993. Interprocedural dynamic slicing applied to interprocedural data flow testing. In ICSM. 386--395.
[95]
Ken Kennedy. 1979. A Survey of Data Flow Analysis Techniques. IBM Thomas J. Watson Research Division.
[96]
Abdelaziz Khamis, Reem Bahgat, and Rana Abdelaziz. 2011. Automatic test data generation using data flow information. Dogus Univ. J. 2 (2011), 140--153.
[97]
Arunkumar Khannur. 2011. Software Testing - Techniques and Applications. Pearson Publications.
[98]
James C. King. 1976. Symbolic execution and program testing. Commun. ACM 19, 7 (July 1976), 385--394.
[99]
Kiran Lakhotia, Mark Harman, and Phil McMinn. 2007. A multi-objective approach to search-based test data generation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1098--1105.
[100]
Kiran Lakhotia, Phil McMinn, and Mark Harman. 2009. Automated test data generation for coverage: Haven’t we solved this problem yet? In Proceedings of the 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques. IEEE Computer Society, 95--104.
[101]
Kiran Lakhotia, Nikolai Tillmann, Mark Harman, and Jonathan de Halleux. 2010. FloPSy - Search-based floating point constraint solving for symbolic execution. In Proceedings of the 22nd IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS’10). 142--157.
[102]
Janusz W. Laski and Bogdan Korel. 1983. A data flow oriented program testing strategy. IEEE Trans. Software Eng. 9, 3 (1983), 347--354.
[103]
Thomas Lengauer and Robert Endre Tarjan. 1979. A fast algorithm for finding dominators in a flowgraph. ACM Trans. Program. Lang. Syst. 1, 1 (1979), 121--141.
[104]
You Li, Zhendong Su, Linzhang Wang, and Xuandong Li. 2013. Steering symbolic execution to less traveled paths. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages 8 Applications (OOPSLA’13), part of SPLASH 2013. 19--32.
[105]
Konstantinos Liaskos and Marc Roper. 2007. Automatic test-data generation: An immunological approach. In Proceedings of Testing: Accademic and Industrial Conference - Practice and Research Techniques (TAIC PART’07). 77--81.
[106]
Konstantinos Liaskos and Marc Roper. 2008. Hybridizing evolutionary testing with artificial immune systems and local search. In Proceedings of the 1st International Conference on Software Testing Verification and Validation (ICST’08), Workshops Proceedings. 211--220.
[107]
Konstantinos Liaskos, Marc Roper, and Murray Wood. 2007. Investigating data-flow coverage of classes using evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1140.
[108]
Yu Lin, Xucheng Tang, Yuting Chen, and Jianjun Zhao. 2009. A divergence-oriented approach to adaptive random testing of java programs. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering (ASE’09). 221--232.
[109]
Chien-Hung Liu, David Chenho Kung, and Pei Hsia. 2000. Object-based data flow testing of web applications. In Proceedings of the 1st Asia-Pacific Conference on Quality Software, 2000. IEEE, 7--16.
[110]
Kin-Keung Ma, Yit Phang Khoo, Jeffrey S. Foster, and Michael Hicks. 2011. Directed symbolic execution. In SAS. 95--111.
[111]
Nicos Malevris and Derek F. Yates. 2006. The collateral coverage of data flow criteria when branch testing. Inf. Softw. Technol. 48, 8 (2006), 676--686.
[112]
Martina Marré and Antonia Bertolino. 1996. Unconstrained duas and their use in achieving all-uses coverage. In Proceedings of the 1996 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’96). ACM, New York, NY, 147--157.
[113]
Martina Marré and Antonia Bertolino. 2003. Using spanning sets for coverage testing. IEEE Trans. Softw. Eng. 29, 11 (Nov. 2003), 974--984.
[114]
Vincenzo Martena, Alessandro Orso, and Mauro Pezzè. 2002. Interclass testing of object oriented software. In Proceedings of the 8th International Conference on Engineering of Complex Computer Systems (ICECCS’02). 135--144.
[115]
Phil McMinn. 2004. Search-based software test data generation: A survey. Softw. Test. Verif. Reliab. 14, 2 (2004), 105--156.
[116]
Lijun Mei, W. K. Chan, and T. H. Tse. 2008. Data flow testing of service-oriented workflow applications. In Proceedings of the ACM/IEEE 30th International Conference on Software Engineering, 2008 (ICSE’08). IEEE, 371--380.
[117]
Lijun Mei, W. K. Chan, and T. H. Tse. 2009. Data flow testing of service choreography. In Proceedings of the the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering. ACM, 151--160.
[118]
Ettore Merlo and Giuliano Antoniol. 1999. A static measure of a subset of intra-procedural data flow testing coverage based on node coverage. In CASCON. 7.
[119]
Zbigniew Michalewicz. 1994. Genetic Algorithms + Data Structures = Evolution Programs (2nd extended ed.). Springer-Verlag New York, New York, NY.
[120]
Matteo Miraz. 2010. Evolutionary Testing of Stateful Systems: A Holistic Approach. Ph.D. Dissertation. Politecnico di Milano.
[121]
Jonathan Misurda, Jim Clause, Juliya Reed, Bruce R. Childers, and Mary Lou Soffa. 2005a. Jazz: A tool for demand-driven structural testing. In Compiler Construction. Springer, 242--245.
[122]
Jonathan Misurda, James A. Clause, Juliya L. Reed, Bruce R. Childers, and Mary Lou Soffa. 2005b. Demand-driven structural testing with dynamic instrumentation. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). 156--165.
[123]
Akbar Siami Namin and James H. Andrews. 2009. The influence of size and coverage on test suite effectiveness. In Proceedings of the 18th International Symposium on Software Testing and Analysis (ISSTA’09). 57--68.
[124]
Narmada Nayak and Durga Prasad Mohapatra. 2010. Automatic test data generation for data flow testing using particle swarm optimization. In IC3 (2). 1--12.
[125]
A. Jefferson Offutt, Zhenyi Jin, and Jie Pan. 1999. The dynamic domain reduction procedure for test data generation. Softw. Pract. Exper. 29, 2 (1999), 167--193.
[126]
Norbert Oster. 2005. Automated generation and evaluation of dataflow-based test data for object-oriented software. In QoSA/SOQUA. 212--226.
[127]
Thomas J. Ostrand and Elaine J. Weyuker. 1991a. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. 74--86.
[128]
Thomas J. Ostrand and Elaine J. Weyuker. 1991b. Data flow-based test adequacy analysis for languages with pointers. In Proceedings of the Symposium on Testing, Analysis, and Verification. ACM, 74--86.
[129]
Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering (ICSE’07). 75--84.
[130]
H. D. Pande, W. A. Landi, and B. G. Ryder. 1994. Interprocedural def-use associations for C systems with single level pointers. IEEE Trans. Softw. Eng. 20, 5 (May 1994), 385--403.
[131]
Mauro Pezzè and Michal Young. 2007. Software Testing and Analysis: Process, Principles and Techniques. Wiley.
[132]
Yu Qi, David Kung, and Eric Wong. 2006. An agent-based data-flow testing approach for Web applications. Inf. Softw. Technol. 48, 12 (2006), 1159--1171.
[133]
Sandra Rapps and Elaine J. Weyuker. 1982. Data flow analysis techniques for test data selection. In Proceedings of the 6th International Conference on Software Engineering (ICSE’82). IEEE Computer Society Press, Los Alamitos, CA, 272--278.
[134]
Sandra Rapps and Elaine J. Weyuker. 1985. Selecting software test data using data flow information. IEEE Trans. Software Eng. 11, 4 (1985), 367--375.
[135]
Torsten Robschink and Gregor Snelting. 2002. Efficient path conditions in dependence graphs. In Proceedings of the 24th International Conference on Software Engineering (ICSE’02). ACM, New York, NY, 478--488.
[136]
Raul Santelices and Mary Jean Harrold. 2007. Efficiently monitoring data-flow test coverage. In Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE’07). ACM, New York, NY, 343--352.
[137]
Raúl A. Santelices, James A. Jones, Yanbing Yu, and Mary Jean Harrold. 2009. Lightweight fault-localization using multiple coverage types. In Proceedings of the 31st International Conference on Software Engineering, (ICSE’09). 56--66.
[138]
Raúl A. Santelices, Saurabh Sinha, and Mary Jean Harrold. 2006. Subsumption of program entities for efficient coverage and monitoring. In Proceedings of the 3rd International Workshop on Software Quality Assurance (SOQUA’06). 2--5.
[139]
Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, New York, NY, 263--272.
[140]
Sanjay Singla, Dharminder Kumar, H. M. Rai, and Priti Singla. 2011a. A hybrid PSO approach to automate test data generation for data flow coverage with dominance concepts. J. Adv. Sci. Technol. 37 (2011), 15--26.
[141]
Sanjay Singla, Priti Singla, and H. M. Rai. 2011b. An automatic test data generation for data flow coverage using soft computing approach. IJRRCS 2, 2 (2011), 265--270.
[142]
Amie L. Souter and Lori L. Pollock. 2003. The construction of contextual def-use associations for object-oriented systems. IEEE Trans. Software Eng. 29, 11 (2003), 1005--1018.
[143]
Ting Su, Zhoulai Fu, Geguang Pu, Jifeng He, and Zhendong Su. 2015. Combining symbolic execution and model checking for data flow testing. In Proceedings of the 37th IEEE/ACM International Conference on Software Engineering (ICSE’15), Volume 1. 654--665.
[144]
Ting Su, Geguang Pu, Bin Fang, Jifeng He, Jun Yan, Siyuan Jiang, and Jianjun Zhao. 2014. Automated coverage-driven test data generation using dynamic symbolic execution. In Proceedings of the 8th International Conference on Software Security and Reliability (SERE’14). 98--107.
[145]
Tao Sun, Zheng Wang, Geguang Pu, Xiao Yu, Zongyan Qiu, and Bin Gu. 2009. Towards scalable compositional test generation. In QSIC. 353--358.
[146]
Paolo Tonella. 2004. Evolutionary testing of classes. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’04). 119--128.
[147]
Hasan Ural, Kassem Saleh, and Alan W. Williams. 2000. Test generation based on control and data dependencies within system specifications in SDL. Comput. Commun. 23, 7 (2000), 609--627.
[148]
Auri Marcelo Rizzo Vincenzi, José Carlos Maldonado, W. Eric Wong, and Márcio Eduardo Delamaro. 2005. Coverage testing of java programs and components. Sci. Comput. Program. 56, 1 (2005), 211--230.
[149]
Mattia Vivanti. 2014. Dynamic data-flow testing. In Proceedings of the 36th International Conference on Software Engineering (ICSE’14), Companion Proceedings. 682--685.
[150]
Mattia Vivanti, Andre Mis, Alessandra Gorla, and Gordon Fraser. 2013. Search-based data-flow test generation. In ISSRE. 370--379.
[151]
Chen-Wei Wang and Alessandra Cavarra. 2009. Checking model consistency using data-flow testing. In Proceedings of the Asia-Pacific Software Engineering Conference, 2009 (APSEC’09). IEEE, 414--421.
[152]
Zheng Wang, Xiao Yu, Tao Sun, Geguang Pu, Zuohua Ding, and Jueliang Hu. 2009. Test data generation for derived types in C program. In TASE. 155--162.
[153]
Joachim Wegener, André Baresel, and Harmen Sthamer. 2001. Evolutionary test environment for automatic structural testing. Inf. Softw. Technol. 43, 14 (2001), 841--854.
[154]
Elaine J. Weyuker. 1990. The cost of data flow testing: An empirical study. IEEE Trans. Software Eng. 16, 2 (1990), 121--128.
[155]
Elaine J. Weyuker. 1993. More experience with data flow testing. IEEE Trans. Software Eng. 19, 9 (1993), 912--919.
[156]
Qian Yang, J. Jenny Li, and David M. Weiss. 2009. A survey of coverage-based testing tools. Comput. J. 52, 5 (2009), 589--597.
[157]
Xiao Yu, Shuai Sun, Geguang Pu, Siyuan Jiang, and Zheng Wang. 2011. A parallel approach to concolic testing with low-cost synchronization. Electr. Notes Theor. Comput. Sci. 274 (2011), 83--96.
[158]
Cristian Zamfir and George Candea. 2010. Execution synthesis: A technique for automated software debugging. In EuroSys. 321--334.
[159]
Jianjun Zhao. 2003. Data-flow-based unit testing of aspect-oriented programs. In Proceedings of the 27th Annual International Computer Software and Applications Conference, 2003 (COMPSAC’03). IEEE, 188--197.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 50, Issue 1
January 2018
588 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3058791
  • Editor:
  • Sartaj Sahni
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2017
Accepted: 01 November 2016
Revised: 01 September 2016
Received: 01 April 2015
Published in CSUR Volume 50, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Data-flow testing
  2. coverage criteria
  3. coverage tracking
  4. data-flow analysis
  5. test data generation

Qualifiers

  • Survey
  • Research
  • Refereed

Funding Sources

  • National Nature Science Foundation of China
  • United States NSF
  • Shanghai STC Project

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)73
  • Downloads (Last 6 weeks)4
Reflects downloads up to 31 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media