survey

Directed Test Generation for Hardware Validation: A Survey

Authors:

Aruna Jayasena,

Prabhat MishraAuthors Info & Claims

ACM Computing Surveys, Volume 56, Issue 5

Article No.: 132, Pages 1 - 36

https://doi.org/10.1145/3638046

Published: 12 January 2024 Publication History

Abstract

The complexity of hardware designs has increased over the years due to the rapid advancement of technology coupled with the need to support diverse and complex features. The increasing design complexity directly translates to difficulty in verifying functional behaviors as well as non-functional requirements. Simulation is the most widely used form of validation using both random and constrained-random test patterns. The random nature of test sequences can cover a vast majority of scenarios, however, it can introduce unacceptable overhead to cover all possible functional and non-functional scenarios. Directed tests are promising to cover the remaining corner cases and hard-to-detect scenarios. Manual development of directed tests can be time-consuming and error-prone. A promising avenue is to perform automated generation of directed tests. In this article, we provide a comprehensive survey of directed test generation techniques for hardware validation. Specifically, we first introduce the complexity of hardware verification to highlight the need for directed test generation. Next, we describe directed test generation using various automated techniques, including formal methods, concolic testing, and machine learning. Finally, we discuss how to effectively utilize the generated test patterns in different validation scenarios, including pre-silicon functional validation, post-silicon debug, as well as validation of non-functional requirements.

References

[1]

Allon Adir, Maxim Golubev, Shimon Landa, Amir Nahir, Gil Shurek, Vitali Sokhin, and Avi Ziv. 2011. Threadmill: A post-silicon exerciser for multi-threaded processors. In Proceedings of the 48th Design Automation Conference. 860–865.

[2]

Alif Ahmed, Farimah Farahmandi, and Prabhat Mishra. 2018. Directed test generation using concolic testing on RTL models. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’18). IEEE, 1538–1543.

[3]

Alif Ahmed and Prabhat Mishra. 2017. QUEBS: Qualifying event based search in concolic testing for validation of RTL models. In Proceedings of the IEEE International Conference on Computer Design (ICCD’17). IEEE, 185–192.

[4]

Tashfia Alam, Zhenkun Yang, Bo Chen, Nicholas Armour, and Sandip Ray. 2022. FirVer: Concolic testing for systematic validation of firmware binaries. In Proceedings of the 27th Asia and South Pacific Design Automation Conference. IEEE, 352–357.

[5]

Peter J. Ashenden. 2010. The Designer’s Guide to VHDL. Morgan Kaufmann.

[6]

Mainak Banga and Michael S. Hsiao. 2008. A region based approach for the identification of hardware Trojans. In Proceedings of the IEEE International Workshop on Hardware-oriented Security and Trust. IEEE, 40–47.

[7]

Kanad Basu, Prabhat Mishra, and Priyadarsan Patra. 2013. Observability-aware directed test generation for soft errors and crosstalk faults. In Proceedings of the 26th International Conference on VLSI Design and 12th International Conference on Embedded Systems. IEEE, 291–296.

[8]

Mike Benjamin, Daniel Geist, Alan Hartman, Gerard Mas, Ralph Smeets, and Yaron Wolfsthal. 1999. A study in coverage-driven test generation. In Proceedings of the 36th Annual ACM/IEEE Design Automation Conference. ACM/IEEE, 970–975.

Digital Library

[9]

Valeria Bertacco. 2010. Post-silicon debugging for multi-core designs. In Proceedings of the 15th Asia and South Pacific Design Automation Conference (ASP-DAC’10). IEEE, 255–258.

[10]

Jayanta Bhadra, Magdy S. Abadir, Li-C. Wang, and Sandip Ray. 2007. A survey of hybrid techniques for functional verification. IEEE Des. Test Comput. 24, 02 (2007), 112–122.

Digital Library

[11]

Jayanta Bhadra, Narayanan Krishnamurthy, and Magdy S. Abadir. 2004. Enhanced equivalence checking: Toward a solidarity of functional verification and manufacturing test generation. IEEE Des. Test Comput. 21, 06 (2004), 494–502.

Digital Library

[12]

K. Uday Bhaskar, M. Prasanth, G. Chandramouli, and V. A. K. V. Kamakoti. 2005. A universal random test generator for functional verification of microprocessors and system-on-chip. In Proceedings of the 18th International Conference on VLSI Design held jointly with the 4th International Conference on Embedded Systems Design. IEEE, 207–212.

Digital Library

[13]

Nicola Bombieri, Franco Fummi, and Valerio Guarnieri. 2011. Accelerating RTL fault simulation through RTL-to-TLM abstraction. In Proceedings of the 16th IEEE European Test Symposium. IEEE, 117–122.

[14]

Nicola Bombieri, Franco Fummi, and Valerio Guarnieri. 2012. FAST: An RTL fault simulation framework based on RTL-To-TLM abstraction. J. Electron. Test. 28, 4 (2012), 495–510.

Digital Library

[15]

Sadullah Canakci, Leila Delshadtehrani, Furkan Eris, Michael Bedford Taylor, Manuel Egele, and Ajay Joshi. 2021. DirectFuzz: Automated test generation for RTL designs using directed graybox fuzzing. In Proceedings of the 58th ACM/IEEE Design Automation Conference (DAC’21). IEEE, 529–534.

Digital Library

[16]

Rajat Subhra Chakraborty, Francis Wolff, Somnath Paul, Christos Papachristou, and Swarup Bhunia. 2009. MERO: A statistical approach for hardware Trojan detection. In Proceedings of the International Workshop on Cryptographic Hardware and Embedded Systems. Springer, 396–410.

Digital Library

[17]

Bo Chen, Kai Cong, Zhenkun Yang, Qin Wang, Jialu Wang, Li Lei, and Fei Xie. 2019. End-to-end concolic testing for hardware/software co-validation. In Proceedings of the IEEE International Conference on Embedded Software and Systems (ICESS’19). IEEE, 1–8.

[18]

Mingsong Chen and Prabhat Mishra. 2011. Decision ordering based property decomposition for functional test generation. In Proceedings of the Design, Automation & Test in Europe Conference. IEEE, 1–6.

[19]

Mingsong Chen, Prabhat Mishra, and Dhrubajyoti Kalita. 2008. Coverage-driven automatic test generation for UML activity diagrams. In Proceedings of the 18th ACM Great Lakes symposium on VLSI. ACM, 139–142.

Digital Library

[20]

Mingsong Chen, Prabhat Mishra, and Dhrubajyoti Kalita. 2012. Automatic RTL test generation from SystemC TLM specifications. ACM Trans. Embed. Comput. Syst. 11, 2 (2012), 1–25.

Digital Library

[21]

Mingsong Chen, Xiaoke Qin, Heon-Mo Koo, and Prabhat Mishra. 2012. System-level Validation: High-level Modeling and Directed Test Generation Techniques. Springer Science & Business Media.

Digital Library

[22]

Mingsong Chen, Xiaoke Qin, and Prabhat Mishra. 2010. Efficient decision ordering techniques for SAT-based test generation. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’10). IEEE, 490–495.

[23]

Kwang-Ting Tim Cheng. 2003. The confluence of manufacturing test and design validation. In Proceedings of the International Test Conference (ITC’03). IEEE Computer Society, 1293–1293.

[24]

Edmund Clarke, Armin Biere, Richard Raimi, and Yunshan Zhu. 2001. Bounded model checking using satisfiability solving. Form. Meth. Syst. Des. 19, 1 (2001), 7–34.

Digital Library

[25]

Edmund M. Clarke and Jeannette M. Wing. 1996. Formal methods: State of the art and future directions. ACM Comput. Surv. 28, 4 (1996), 626–643.

Digital Library

[26]

Kai Cong, Fei Xie, and Li Lei. 2013. Automatic concolic test generation with virtual prototypes for post-silicon validation. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’13). IEEE, 303–310.

[27]

Jonathan Cruz, Farimah Farahmandi, Alif Ahmed, and Prabhat Mishra. 2018. Hardware Trojan detection using ATPG and model checking. In Proceedings of the 31st International Conference on VLSI Design and 17th International Conference on Embedded Systems (VLSID’18). IEEE, 91–96.

[28]

Thanh Nga Dang, Abhik Roychoudhury, Tulika Mitra, and Prabhat Mishra. 2009. Generating test programs to cover pipeline interactions. In Proceedings of the 46th ACM/IEEE Design Automation Conference. IEEE, 142–147.

[29]

Shujun Deng, Kwang-Ting Cheng, Jinian Bian, and Zhiqiu Kong. 2010. Mutation-based diagnostic test generation for hardware design error diagnosis. In Proceedings of the IEEE International Test Conference, Vol. 1. IEEE Computer Society, 1–1.

[30]

Andrew DeOrio, Daya Shanker Khudia, and Valeria Bertacco. 2011. Post-silicon bug diagnosis with inconsistent executions. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’11). IEEE, 755–761.

[31]

Paul E. Dodd and Lloyd W. Massengill. 2003. Basic mechanisms and modeling of single-event upset in digital microelectronics. IEEE Trans. Nuclear Sci. 50, 3 (2003), 583–602.

[32]

Jack Edmonds and Ellis L. Johnson. 1973. Matching, Euler tours and the Chinese postman. Math. Program. 5 (1973), 88–124.

Digital Library

[33]

Bernhard Egger, Eunjin Song, Hochan Lee, and Daeyong Shin. 2019. Random test program generation for verification and validation of the Samsung Reconfigurable Processor. J. Syst. Archit. 97 (2019), 219–238.

Digital Library

[34]

Marco Elver and Vijay Nagarajan. 2016. McVerSi: A test generation framework for fast memory consistency verification in simulation. In Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA’16). IEEE, 618–630.

[35]

Nestoras E. Evmorfopoulos, Georgios I. Stamoulis, and John N. Avaritsiotis. 2002. A Monte Carlo approach for maximum power estimation based on extreme value theory. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 21, 4 (2002), 415–432.

Digital Library

[36]

Farimah Farahmandi and Prabhat Mishra. 2016. Automated test generation for debugging arithmetic circuits. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’16). IEEE, 1351–1356.

[37]

Farimah Farahmandi, Prabhat Mishra, and Sandip Ray. 2016. Exploiting transaction level models for observability-aware post-silicon test generation. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’16). IEEE, 1477–1480.

[38]

Harry Foster. 2020. 2020 Wilson research group functional verification study: IC/ASIC functional verification trend report. Wilson Research Group and Mentor, A Siemens Business, White Paper (2020). https://resources.sw.siemens.com/en-US/white-paper-2020-wilson-research-group-functional-verification-study-ic-asic-fucntional-verification-trend-report

[39]

Nikos Foutris, Dimitris Gizopoulos, Mihalis Psarakis, Xavier Vera, and Antonio Gonzalez. 2011. Accelerating microprocessor silicon validation by exposing ISA diversity. In Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture. 386–397.

Digital Library

[40]

Ana T. Freitas, Horácio C. Neto, and Arlindo L. Oliveira. 2000. On the complexity of power estimation problems. In Proceedings of the International Workshop on Logic Synthesis (ILWS’00). 239–244.

[41]

Franco Fummi. 2003. The confluence of manufacturing test and design validation. In Proceedings of the International Test Conference (ITC’03). IEEE Computer Society, 1291–1291.

[42]

Daniel Geist, Monica Farkas, Avner Landver, Yossi Lichtenstein, Shmuel Ur, and Yaron Wolfsthal. 1996. Coverage-directed test generation using symbolic techniques. In Proceedings of the International Conference on Formal Methods in Computer-aided Design. Springer, 143–158.

[43]

Frank Ghenassia, Alain Clouard, Laurent Maillet-contoz, Jean-philippe Strassen, Eric Paire, Thibaut Bultiaux, Stephane Guenot, Serge Hustin, Alexandre Blampey, Joseph Bulone, Matthieu Moy, Antoine Perrin, Gregory Poivre, Christophe Amerijckx, and Amine Kerkeni. 2005. Transaction-level Modeling with SystemC. Vol. 2. Springer.

[44]

Google Fuzz Tester. Google Fuzz Tester. 2023. Retrieved from https://google.github.io/oss-fuzz/

[45]

Tomás Grimm, Djones Lettnin, and Michael Hübner. 2018. A survey on formal verification techniques for safety-critical systems-on-chip. Electronics 7, 6 (2018), 81.

[46]

Aarti Gupta. 1992. Formal hardware verification methods: A survey. In Computer-aided Verification. Springer, 5–92.

[47]

Onur Guzey and Li-C. Wang. 2007. Coverage-directed test generation through automatic constraint extraction. In Proceedings of the IEEE International High Level Design Validation and Test Workshop. IEEE, 151–158.

[48]

Hadi Hajimiri, Kamran Rahmani, and Prabhat Mishra. 2015. Efficient peak power estimation using probabilistic cost-benefit analysis. In Proceedings of the 28th International Conference on VLSI Design. IEEE, 369–374.

[49]

Ian G. Harris. 2003. The confluence of manufacturing test and design validation. In Proceedings of the International Test Conference (ITC’03). IEEE Computer Society, 1290–1290.

[50]

Miao He, Jungmin Park, Adib Nahiyan, Apostol Vassilev, Yier Jin, and Mark Tehranipoor. 2019. RTL-PSC: Automated power side-channel leakage assessment at register-transfer level. In Proceedings of the IEEE VLSI Test Symposium (VTS’19). 1–6.

[51]

Ted Hong, Yanjing Li, Sung-Boem Park, Diana Mui, David Lin, Ziyad Abdel Kaleq, Nagib Hakim, Helia Naeimi, Donald S. Gardner, and Subhasish Mitra. 2010. QED: Quick error detection tests for effective post-silicon validation. In Proceedings of the IEEE International Test Conference. IEEE, 1–10.

[52]

Yuanwen Huang, Swarup Bhunia, and Prabhat Mishra. 2016. MERS: Statistical test generation for side-channel analysis based Trojan detection. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 130–141.

[53]

Jaewon Hur, Suhwan Song, Dongup Kwon, Eunjin Baek, Jangwoo Kim, and Byoungyoung Lee. 2021. DIFUZZRTL: Differential fuzz testing to find CPU bugs. In Proceedings of the IEEE Symposium on Security and Privacy (SP’21). IEEE, 1286–1303.

[54]

Charalambos Ioannides and Kerstin I. Eder. 2012. Coverage-directed test generation automated by machine learning–a review. ACM Trans. Des. Autom. Electron. Syst. 17, 1 (2012), 1–21.

Digital Library

[55]

Aruna Jayasena, Emma Andrews, and Prabhat Mishra. 2023. TVLA*: Test vector leakage assessment on hardware implementations of asymmetric cryptography algorithms. IEEE Trans. Very Large Scale Integ. Syst. 31, 9 (2023).

[56]

Aruna Jayasena, Binod Kumar, Subodha Charles, Hasini Witharana, and Prabhat Mishra. 2022. Network-on-chip trust validation using security assertions. J. Hardw. Syst. Secur. 6 (2022), 79–94.

[57]

Aruna Jayasena and Prabhat Mishra. 2023. HIVE: Scalable hardware-firmware co-verification using scenario-based decomposition and automated hint extraction. arXiv preprint arXiv:2309.08002 (2023).

[58]

Aruna Jayasena and Prabhat Mishra. 2023. Scalable detection of hardware trojans using ATPG-based activation of rare events. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 42, 12 (2023).

[59]

Seonghun Jeong, Youngchul Cho, Daeyong Shin, Changyeon Jo, Yenjo Han, Soojung Ryu, Jeongwook Kim, and Bernhard Egger. 2012. Random test program generation for reconfigurable architectures. In Proceedings of the 13th International Workshop on Microprocessor Test and Verification (MTV’12).

[60]

Christoph Kern and Mark R. Greenstreet. 1999. Formal verification in hardware design: A survey. ACM Trans. Des. Autom. Electron. Syst. 4, 2 (1999), 123–193.

Digital Library

[61]

Nathan Kitchen and Andreas Kuehlmann. 2007. Stimulus generation for constrained random simulation. In Proceedings of the IEEE/ACM International Conference on Computer-aided Design. IEEE, 258–265.

[62]

Tom Kolan, Hillel Mendelson, Vitali Sokhin, Shai Doron, Hernan Theiler, Shay Aviv, Hagai Hadad, Natalia Freidman, Elena Tsanko, John Ludden, and Bryant Cockcroft. 2021. Post silicon validation of the MMU. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’21). IEEE, 212–217.

[63]

Heon-Mo Koo and Prabhat Mishra. 2006. Functional test generation using property decompositions for validation of pipelined processors. In Proceedings of the Design Automation & Test in Europe Conference, Vol. 1. IEEE, 1–6.

[64]

Heon-Mo Koo and Prabhat Mishra. 2006. Test generation using SAT-based bounded model checking for validation of pipelined processors. In Proceedings of the 16th ACM Great Lakes Symposium on VLSI. 362–365.

Digital Library

[65]

Heon-Mo Koo and Prabhat Mishra. 2008. Specification-based compaction of directed tests for functional validation of pipelined processors. In Proceedings of the 6th IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis. 137–142.

Digital Library

[66]

Heon-Mo Koo, Prabhat Mishra, Jayanta Bhadra, and Magdy Abadir. 2006. Directed micro-architectural test generation for an industrial processor: A case study. In Proceedings of the 7th International Workshop on Microprocessor Test and Verification (MTV’06). IEEE, 33–36.

[67]

Harish Kriplani, Farid Najm, and Ibrahim Hajj. 1992. Maximum current estimation in CMOS circuits. In Proceedings of the 29th ACM/IEEE Design Automation Conference. IEEE, 2–7.

Digital Library

[68]

Arun Krishnamachary, Jacob A. Abraham, and Raghuram S. Tupuri. 2001. Timing verification and delay test generation for hierarchical designs. In Proceedings of the 14th International Conference on VLSI Design. IEEE, 157–162.

[69]

Thomas Kropf and Hans-Joachim Wunderlich. 1991. A common approach to test generation and hardware verification based on temporal logic. In Proceedings of the International Test Conference. IEEE, 57–66.

[70]

Kevin Laeufer, Jack Koenig, Donggyu Kim, Jonathan Bachrach, and Koushik Sen. 2018. RFUZZ: Coverage-directed fuzz testing of RTL on FPGAs. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’18). IEEE, 1–8.

[71]

Nicole Lesperance, Shrikant Kulkarni, and Kwang-Ting Cheng. 2015. Hardware Trojan detection using exhaustive testing of k-bit subspaces. In Proceedings of the 20th Asia and South Pacific Design Automation Conference. IEEE, 755–760.

[72]

Rongjian Liang, Nathaniel Pinckney, Yuji Chai, Haoxin Ren, and Brucek Khailany. 2023. Late breaking results: Test selection for RTL coverage by unsupervised learning from fast functional simulation. In Proceedings of the 60th ACM/IEEE Design Automation Conference (DAC’23). IEEE, 1–2.

[73]

David Lin, Ted Hong, Farzan Fallah, Nagib Hakim, and Subhasish Mitra. 2012. Quick detection of difficult bugs for effective post-silicon validation. In Proceedings of the Design Automation Conference (DAC’12). IEEE, 561–566.

[74]

Lingyi Liu and Shabha Vasudevan. 2009. STAR: Generating input vectors for design validation by static analysis of RTL. In Proceedings of the IEEE International High Level Design Validation and Test Workshop. IEEE, 32–37.

[75]

Lingyi Liu and Shobha Vasudevan. 2011. Efficient validation input generation in RTL by hybridized source code analysis. In Proceedings of the Design, Automation & Test in Europe Conference. IEEE, 1–6.

[76]

Lingyi Liu and Shobha Vasudevan. 2014. Scaling input stimulus generation through hybrid static and dynamic analysis of RTL. ACM Trans. Des. Autom. Electron. Syst. 20, 1 (2014), 1–33.

Digital Library

[77]

Jason Lowe-Power. 2023. gem5: Simulator system. Retrieved from https://www.gem5.org

[78]

David J. Lu. 1982. Watchdog processors and structural integrity checking. IEEE Trans. Comput. 31, 07 (1982), 681–685.

Digital Library

[79]

Yangdi Lyu, Alif Ahmed, and Prabhat Mishra. 2019. Automated activation of multiple targets in RTL models using concolic testing. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’19). IEEE, 354–359.

[80]

Yangdi Lyu and Prabhat Mishra. 2019. Efficient test generation for Trojan detection using side channel analysis. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’19). IEEE, 408–413.

[81]

Yangdi Lyu and Prabhat Mishra. 2020. Automated test generation for Trojan detection using delay-based side channel analysis. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’20). IEEE, 1031–1036.

[82]

Yangdi Lyu and Prabhat Mishra. 2020. Scalable activation of rare triggers in hardware trojans by repeated maximal clique sampling. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 40, 7 (2020), 1287–1300.

[83]

Yangdi Lyu and Prabhat Mishra. 2021. MaxSense: Side-channel sensitivity maximization for trojan detection using statistical test patterns. ACM Trans. Des. Autom. Electron. Syst. 26, 3 (2021), 1–21.

Digital Library

[84]

Yangdi Lyu, Xiaoke Qin, Mingsong Chen, and Prabhat Mishra. 2018. Directed test generation for validation of cache coherence protocols. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 38, 1 (2018), 163–176.

Digital Library

[85]

Aamer Mahmood and Edward J. McCluskey. 1988. Concurrent error detection using watchdog processors—A survey. IEEE Trans. Comput. 37, 2 (1988), 160–174.

Digital Library

[86]

Microsoft Fuzz Tester. Microsoft Fuzz Tester. 2023. Retrieved from https://github.com/microsoft/onefuzz. August 31, 2023.

[87]

Prabhat Mishra, Swarup Bhunia, and Mark Tehranipoor. 2017. Hardware IP Security and Trust. Springer.

[88]

Prabhat Mishra and Mingsong Chen. 2009. Efficient techniques for directed test generation using incremental satisfiability. In Proceedings of the 22nd International Conference on VLSI Design. IEEE, 65–70.

[89]

Prabhat Mishra and Nikil Dutt. 2008. Specification-driven directed test generation for validation of pipelined processors. ACM Trans. Des. Autom. Electron. Syst. 13, 3 (2008), 1–36.

Digital Library

[90]

Prabhat Mishra and Farimah Farahmandi. 2019. Post-silicon Validation and Debug. Springer.

[91]

Prabhat Mishra, Ronny Morad, Avi Ziv, and Sandip Ray. 2017. Post-silicon validation in the SoC era: A tutorial introduction. IEEE Des. Test 34, 3 (2017), 68–92.

[92]

Matthew W. Moskewicz, Conor F. Madigan, Ying Zhao, Lintao Zhang, and Sharad Malik. 2001. Chaff: Engineering an efficient SAT solver. In Proceedings of the 38th Annual Design Automation Conference. 530–535.

Digital Library

[93]

Rajdeep Mukherjee, Daniel Kroening, and Tom Melham. 2015. Hardware verification using software analyzers. In Proceedings of the IEEE Computer Society Annual Symposium on VLSI. IEEE, 7–12.

[94]

K. Najeeb, Karthik Gururaj, V. Kamakoti, and Vivekanand M. Vedula. 2007. Controllability-driven power virus generation for digital circuits. In Proceedings of the 20th International Conference on VLSI Design held jointly with the 6th International Conference on Embedded Systems (VLSID’07). IEEE, 407–412.

Digital Library

[95]

Seetharam Narasimhan, Dongdong Du, Rajat Subhra Chakraborty, Somnath Paul, Francis G. Wolff, Christos A. Papachristou, Kaushik Roy, and Swarup Bhunia. 2012. Hardware Trojan detection by multiple-parameter side-channel analysis. IEEE Trans. Comput. 62, 11 (2012), 2183–2195.

Digital Library

[96]

Seetharam Narasimhan, Xinmu Wang, Dongdong Du, Rajat Subhra Chakraborty, and Swarup Bhunia. 2011. TeSR: A robust temporal self-referencing approach for hardware Trojan detection. In Proceedings of the IEEE International Symposium on Hardware-oriented Security and Trust. IEEE, 71–74.

[97]

Yehuda Naveh, Michal Rimon, Itai Jaeger, Yoav Katz, Michael Vinov, Eitan S. Marcu, and Gil Shurek. 2007. Constraint-based random stimuli generation for hardware verification. AI Mag. 28, 3 (2007), 13–13.

Digital Library

[98]

Chris Nigh and Alex Orailoglu. 2021. AdaTrust: Combinational hardware trojan detection through adaptive test pattern construction. IEEE Trans. Very Large Scale Integ. Syst. 29, 3 (2021), 544–557.

[99]

Yann Oddos, Katell Morin-Allory, Dominique Borrione, Marc Boulé, and Zeljko Zilic. 2009. MYGEN: Automata-based on-line test generator for assertion-based verification. In Proceedings of the 19th ACM Great Lakes Symposium on VLSI. 75–80.

[100]

Nahmsuk Oh, Philip P. Shirvani, and Edward J. McCluskey. 2002. Error detection by duplicated instructions in super-scalar processors. IEEE Trans. Reliab. 51, 1 (2002), 63–75.

[101]

Zhixin Pan and Prabhat Mishra. 2021. Automated test generation for hardware trojan detection using reinforcement learning. In Proceedings of the 26th Asia and South Pacific Design Automation Conference. 408–413.

Digital Library

[102]

Zhixin Pan, Jennifer Sheldon, and Prabhat Mishra. 2020. Test generation using reinforcement learning for delay-based side-channel analysis. In Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD’20). IEEE, 1–7.

[103]

George Papadimitriou, Athanasios Chatzidimitriou, Dimitris Gizopoulos, and Ronny Morad. 2016. An agile post-silicon validation methodology for the address translation mechanisms of modern microprocessors. IEEE Trans. Device Mater. Reliab. 17, 1 (2016), 3–11.

[104]

George Papadimitriou, Dimitris Gizopoulos, Athanasios Chatzidimitriou, Tom Kolan, Anatoly Koyfman, Ronny Morad, and Vitali Sokhin. 2016. Unveiling difficult bugs in address translation caching arrays for effective post-silicon validation. In Proceedings of the IEEE 34th International Conference on Computer Design (ICCD’16). IEEE, 544–551.

[105]

Irith Pomeranz and Sudhakar M. Reddy. 2004. A measure of quality for n-detection test sets. IEEE Trans. Comput. 53, 11 (2004), 1497–1503.

Digital Library

[106]

Nitin Pundir, Jungmin Park, Farimah Farahmandi, and Mark Tehranipoor. 2022. Power side-channel leakage assessment framework at register-transfer level. IEEE Trans. Very Large Scale Integ. Syst. 30, 9 (2022).

[107]

Xiaoke Qin, Mingsong Chen, and Prabhat Mishra. 2010. Synchronized generation of directed tests using satisfiability solving. In Proceedings of the 23rd International Conference on VLSI Design. IEEE, 351–356.

[108]

Xiaoke Qin and Prabhat Mishra. 2012. Automated generation of directed tests for transition coverage in cache coherence protocols. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE’22). IEEE, 3–8.

[109]

Xiaoke Qin and Prabhat Mishra. 2012. Directed test generation for validation of multicore architectures. ACM Trans. Des. Autom. Electron. Syst. 17, 3 (2012), 1–21.

Digital Library

[110]

Xiaoke Qin and Prabhat Mishra. 2014. Scalable test generation by interleaving concrete and symbolic execution. In Proceedings of the 27th International Conference on VLSI Design and 13th International Conference on Embedded Systems. IEEE, 104–109.

[111]

Xiaoke Qin, Weixun Wang, and Prabhat Mishra. 2012. TCEC: Temperature and energy-constrained scheduling in real-time multitasking systems. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 31, 8 (2012), 1159–1168.

Digital Library

[112]

Danilo Ravotto, Ernesto Sánchez, Massimiliano Schillaci, and Giovanni Squillero. 2008. An evolutionary methodology for test generation for peripheral cores via dynamic FSM extraction. In Proceedings of the Workshops on Applications of Evolutionary Computation. Springer, 214–223.

[113]

Mohammad Sabri, Ahmad Shabani, and Bijan Alizadeh. 2021. SAT-based integrated hardware trojan detection and localization approach through path-delay analysis. IEEE Trans. Circ. Syst. II: Express Briefs 68, 8 (2021), 2850–2854.

[114]

Sayandeep Saha, Rajat Subhra Chakraborty, Srinivasa Shashank Nuthakki, and Debdeep Mukhopadhyay. 2015. Improved test pattern generation for hardware trojan detection using genetic algorithm and Boolean satisfiability. In Proceedings of the International Workshop on Cryptographic Hardware and Embedded Systems. Springer, 577–596.

[115]

Jacob Savir. 1997. Delay test generation: A hardware perspective. J. Electron. Test. 10, 3 (1997), 245–254.

Digital Library

[116]

Yiannakis Sazeides, Alex Gerber, Ron Gabor, Arkady Bramnik, George Papadimitriou, Dimitris Gizopoulos, Chrysostomos Nicopoulos, Giorgos Dimitrakopoulos, and Karyofyllis Patsidis. 2022. IDLD: Instantaneous detection of leakage and duplication of identifiers used for register renaming. In Proceedings of the 55th IEEE/ACM International Symposium on Microarchitecture (MICRO’22). IEEE, 799–814.

Digital Library

[117]

Divjyot Sethi, Muralidhar Talupur, and Sharad Malik. 2014. Using flow specifications of parameterized cache coherence protocols for verifying deadlock freedom. In Automated Technology for Verification and Analysis. Springer, 330–347.

[118]

Zhendong Shi, Haocheng Ma, Qizhi Zhang, Yanjiang Liu, Yiqiang Zhao, and Jiaji He. 2021. Test generation for hardware trojan detection using correlation analysis and genetic algorithm. ACM Trans. Embed. Comput. Syst. 20, 4 (2021), 1–20.

Digital Library

[119]

Douglas J. Smith. 1996. VHDL and Verilog compared and contrasted-plus modeled example written in VHDL, Verilog and C. In Proceedings of the 33rd Design Automation Conference (DAC’96). IEEE, 771–776.

Digital Library

[120]

Paul Stephan, Robert K. Brayton, and Alberto L. Sangiovanni-Vincentelli. 1996. Combinational test generation using satisfiability. IEEE Trans. Comput.-aid. Des. Integ. Circ. Syst. 15, 9 (1996), 1167–1176.

Digital Library

[121]

Donald Thomas and Philip Moorby. 2008. The Verilog® Hardware Description Language. Springer Science & Business Media.

Digital Library

[122]

Jason G. Tong, Marc Boulé, and Zeljko Zilic. 2013. Test compaction techniques for assertion-based test generation. ACM Trans. Des. Autom. Electron. Syst. 19, 1 (2013), 1–29.

Digital Library

[123]

Shobha Vasudevan, Wenjie Joe Jiang, David Bieber, Rishabh Singh, C. Richard Ho, and Charles Sutton. 2021. Learning semantic representations to verify hardware designs. Adv. Neural Inf. Process. Syst. 34 (2021), 23491–23504.

[124]

Freek Verbeek, Pooria M. Yaghini, Ashkan Eghbal, and Nader Bagherzadeh. 2016. Deadlock verification of cache coherence protocols and communication fabrics. IEEE Trans. Comput. 66, 2 (2016), 272–284.

[125]

Manisha Vinta and S. Sivanantham. 2020. Modeling and test generation for combinational hardware trojans. In Proceedings of the IEEE International Test Conference India. IEEE, 1–4.

[126]

Ilya Wagner and Valeria Bertacco. 2008. MCjammer: Adaptive verification for multi-core designs. In Proceedings of the Conference on Design, Automation and Test in Europe. 670–675.

[127]

Ilya Wagner and Valeria Bertacco. 2008. Reversi: Post-silicon validation system for modern microprocessors. In Proceedings of the IEEE International Conference on Computer Design. IEEE, 307–314.

[128]

Chuan-Yu Wang and Kaushik Roy. 1996. Maximum power estimation for CMOS circuits using deterministic and statistic approaches. In Proceedings of the 9th International Conference on VLSI Design. IEEE, 364–369.

[129]

Chuan-Yu Wang and Kaushik Roy. 1997. Estimation of maximum power for sequential circuits considering spurious transitions. In Proceedings of the International Conference on Computer Design VLSI in Computers and Processors. IEEE, 746–751.

[130]

Weixun Wang, Xiaoke Qin, and Prabhat Mishra. 2010. Temperature-and energy-constrained scheduling in multitasking systems: A model checking approach. In Proceedings of the ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED’10). IEEE, 85–90.

[131]

Hasini Witharana, Yangdi Lyu, Subodha Charles, and Prabhat Mishra. 2022. A survey on assertion-based hardware verification. ACM Comput. Surv. 54, 11s (2022), 1–33.

Digital Library

[132]

Hasini Witharana, Yangdi Lyu, and Prabhat Mishra. 2021. Directed test generation for activation of security assertions in RTL models. ACM Trans. Des. Autom. Electron. Syst. 26, 4 (2021), 1–28.

Digital Library

[133]

David A. Wood, Garth A. Gibson, and Randy H. Katz. 1990. Verifying a multiprocessor cache controller using random test generation. IEEE Des. Test Comput. 7, 4 (1990), 13–25.

Digital Library

[134]

Meng Zhang, Jesse D. Bingham, John Erickson, and Daniel J. Sorin. 2014. PVCoherence: Designing flat coherence protocols for scalable verification. In Proceedings of the International Symposium on High Performance Computer Architecture (HPCA’14). IEEE, 392–403.

[135]

Tao Zhang, Jungmin Park, Mark Tehranipoor, and Farimah Farahmandi. 2021. PSC-TG: RTL power side-channel leakage assessment with test pattern generation. In Proceedings of the ACM/IEEE Design Automation Conference (DAC’21). 709–714.

Cited By

Cicirello V(2024)Evolutionary Computation: Theories, Techniques, and ApplicationsApplied Sciences10.3390/app1406254214:6(2542)Online publication date: 18-Mar-2024
https://doi.org/10.3390/app14062542
Jayasena ABachmann RMishra P(2024)EvilCS: An Evaluation of Information Leakage through Context Switching on Security Enclaves2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546809(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546809
Mulhem SEwert CNešković APoudel A(2024)Secure Software/Hardware Hybrid In-Field Testing for System-on-Chip2024 IFIP/IEEE 32nd International Conference on Very Large Scale Integration (VLSI-SoC)10.1109/VLSI-SoC62099.2024.10767817(1-6)Online publication date: 6-Oct-2024
https://doi.org/10.1109/VLSI-SoC62099.2024.10767817
Show More Cited By

Index Terms

Directed Test Generation for Hardware Validation: A Survey
1. Hardware
  1. Hardware test
    1. Test-pattern generation and fault simulation
  2. Hardware validation
    1. Functional verification
    2. Post-manufacture validation and debug
      1. Bug detection, localization and diagnosis

Recommendations

A Survey on Assertion-based Hardware Verification
Hardware verification of modern electronic systems has been identified as a major bottleneck due to the increasing complexity and time-to-market constraints. One of the major objectives in hardware verification is to drastically reduce the validation and ...
Specification-driven directed test generation for validation of pipelined processors

Functional validation is a major bottleneck in pipelined processor design due to the combined effects of increasing design complexity and lack of efficient techniques for directed test generation. Directed test vectors can reduce overall validation ...
Test generation using SAT-based bounded model checking for validation of pipelined processors
GLSVLSI '06: Proceedings of the 16th ACM Great Lakes symposium on VLSI

Functional verification is one of the major bottlenecks in microprocessor design. Simulation-based techniques are the most widely used form of processor verification. Efficient test generation is crucial for the simulation-based verification. We present ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 56, Issue 5

May 2024

1019 pages

EISSN:1557-7341

DOI:10.1145/3613598

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 January 2024

Online AM: 19 December 2023

Accepted: 15 December 2023

Revised: 30 October 2023

Received: 08 February 2023

Published in CSUR Volume 56, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey

Funding Sources

National Science Foundation (NSF)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
918
Total Downloads

Downloads (Last 12 months)918
Downloads (Last 6 weeks)131

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cicirello V(2024)Evolutionary Computation: Theories, Techniques, and ApplicationsApplied Sciences10.3390/app1406254214:6(2542)Online publication date: 18-Mar-2024
https://doi.org/10.3390/app14062542
Jayasena ABachmann RMishra P(2024)EvilCS: An Evaluation of Information Leakage through Context Switching on Security Enclaves2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546809(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546809
Mulhem SEwert CNešković APoudel A(2024)Secure Software/Hardware Hybrid In-Field Testing for System-on-Chip2024 IFIP/IEEE 32nd International Conference on Very Large Scale Integration (VLSI-SoC)10.1109/VLSI-SoC62099.2024.10767817(1-6)Online publication date: 6-Oct-2024
https://doi.org/10.1109/VLSI-SoC62099.2024.10767817
Jayasena AMishra P(2024)HIVE: Scalable Hardware-Firmware Co-Verification Using Scenario-Based Decomposition and Automated Hint ExtractionIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.338396143:10(3278-3291)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TCAD.2024.3383961
Gadde DKumar ALettnin DSimon S(2024)FuzzWiz - Fuzzing Framework for Efficient Hardware Coverage2024 International Symposium on Electronics and Telecommunications (ISETC)10.1109/ISETC63109.2024.10797245(1-5)Online publication date: 7-Nov-2024
https://doi.org/10.1109/ISETC63109.2024.10797245
Xia SZhang YWang ZDing RCui HChen X(2024)An Approach to Enhance the Efficiency of RISC-V Verification Using Intelligent Algorithms2024 IEEE 7th International Conference on Electronic Information and Communication Technology (ICEICT)10.1109/ICEICT61637.2024.10670953(419-423)Online publication date: 31-Jul-2024
https://doi.org/10.1109/ICEICT61637.2024.10670953

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents