Toss a Fault to BpfChecker: Revealing Implementation Flaws for eBPF runtimes with Differential Fuzzing
Pages 3928 - 3942
Abstract
eBPF is a revolutionary technology that can run sandboxed programs in a privileged context and has an extensive range of applications, such as network monitoring on Linux kernel, denial-of-service protection on Windows, and the execution mechanism of smart contracts on blockchain. However, implementation flaws in eBPF have broad-reaching impact and serious consequences. Prior studies primarily focus on the memory safety of the eBPF runtimes, but few can detect implementation flaws (i.e., whether the implementation is correct). Meanwhile, existing implementation flaws detecting methods predominantly address bugs in the verifier, neglecting bugs in other components (i.e., the interpreter and the JIT compiler). In this paper, we present BpfChecker, a differential fuzzing framework to detect implementation flaws in the eBPF runtimes. It utilizes eBPF programs as input, performing differential testing for the critical states across various eBPF runtimes to uncover implementation flaws. To enhance the semantics of generated programs, we devise a lightweight intermediate representation and perform constrained mutations under the guidance of error messages. We have implemented a prototype of BpfChecker and extensively evaluated it on the three eBPF runtimes (i.e., Solana rBPF, vanilla rBPF, Windows eBPF). As a result, we have uncovered 28 new implementation flaws, received 2 CVEs and 800,000 bounty with developers' acknowledgment. More importantly, 2 of the newly found bugs can be used to create divergences in the execution layer of the Solana network.
References
[1]
Marcelo Abranches, Oliver Michel, Eric Keller, and Stefan Schmid. 2021. Efficient network monitoring applications in the kernel with ebpf and xdp. In 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN). IEEE, 28--34.
[2]
Maher Alharby, Amjad Aldweesh, and Aad Van Moorsel. 2018. Blockchain-based smart contracts: A systematic mapping study of academic research (2018). In 2018 International Conference on Cloud Computing, Big Data and Blockchain (ICCBB). IEEE, 1--6.
[3]
Peng Chen and Hao Chen. 2018. Angora: Efficient fuzzing by principled search. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 711--725.
[4]
Alexei Starovoitov Daniel Borkmann. 2024. eBPF. https://ebpf.io
[5]
Poorna Gaddehosur Dave Thaler. 2021. Making eBPF work on Windows. https://cloudblogs.microsoft.com/opensource/2021/05/10/making-ebpf-work-on-windows Retrieved March 1, 2024 from
[6]
Zizhuang Deng, Guozhu Meng, Kai Chen, Tong Liu, Lu Xiang, and Chunyang Chen. 2023. Differential Testing of Cross Deep Learning Framework APIs: Revealing Inconsistencies and Vulnerabilities. In 32nd USENIX Security Symposium (USENIX Security 23). 7393--7410.
[7]
Andrey Konovalov. Dmitry Vyukov. 2024. Syzkaller: an unsupervised coverage-guided kernel fuzzer. https://github.com/google/syzkaller
[8]
Elazar Gershuni, Nadav Amit, Arie Gurfinkel, Nina Narodytska, Jorge A Navas, Noam Rinetzky, Leonid Ryzhyk, and Mooly Sagiv. 2019. Simple and precise static analysis of untrusted linux kernel extensions. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 1069--1084.
[9]
Google. 2024. Buzzer - An eBPF Fuzzer toolchain. https://github.com/google/buzzer
[10]
Alex Groce, Mohammad Amin Alipour, and Rahul Gopinath. 2014. Coverage and its discontents. In Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software. 255--268.
[11]
Hsin-Wei Hung and Ardalan Amiri Sani. 2023. BRF: eBPF Runtime Fuzzer. arXiv preprint arXiv:2305.08782 (2023).
[12]
Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren. 2022. EXAMINER: Automatically locating inconsistent instructions between real devices and CPU emulators for ARM. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 846--858.
[13]
Linux kernel. 2024. eBPF verifier ? The Linux Kernel documentation.
[14]
Jung-Bok Lee, Tae-Hee Yoo, Eo-Hyung Lee, Byeong-Ha Hwang, Sung-Won Ahn, and Choong-Hee Cho. 2021. High-performance software load balancer for cloud-native architecture. IEEE Access, Vol. 9 (2021), 123704--123716.
[15]
Youlin Li, Weina Niu, Yukun Zhu, Jiacheng Gong, Beibei Li, and Xiaosong Zhang. 2023. Fuzzing Logical Bugs in eBPF Verifier with Bound-Violation Indicator. In ICC 2023-IEEE International Conference on Communications. IEEE, 753--758.
[16]
William M McKeeman. 1998. Differential testing for software. Digital Technical Journal, Vol. 10, 1 (1998), 100--107.
[17]
Microsoft. 2024. eBPF implementation that runs on top of Windows. https://github.com/microsoft/ebpf-for-windows
[18]
Microsoft. 2024. Microsoft Security Response Center. https://msrc.microsoft.com
[19]
Microsoft. 2024. microsoft/retina: eBPF distributed networking observability tool for Kubernetes.
[20]
Microsoft. 2024 d. Windows eBPF project fuzzer. https://github.com/microsoft/ebpf-for-windows/tree/b9d6cb6b7edcc5314413d866a63d36ebc41ab14d/tests/libfuzzer
[21]
MITRE. 2020. CVE-2020--8835. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020--8835 Retrieved March 1, 2024 from
[22]
MITRE. 2022. CVE-2022--23066. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022--23066 Retrieved March 1, 2024 from
[23]
MITRE. 2023. CVE - CVE-2023--2163. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023--2163 Retrieved March 1, 2024 from
[24]
MITRE. 2024. CVE - CVE-2021--46102. https://cve.mitre.org/cgi-bin/cvename.cgi?name=2021--46102 Retrieved March 1, 2024 from
[25]
MITRE. 2024. CVE - CVE-2024--26588. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024--26588 Retrieved March 1, 2024 from
[26]
Luke Nelson, James Bornholt, Ronghui Gu, Andrew Baumann, Emina Torlak, and Xi Wang. 2019. Scaling symbolic evaluation for automated verification of systems code with Serval. In Proceedings of the 27th ACM Symposium on Operating Systems Principles. 225--242.
[27]
Luke Nelson, Jacob Van Geffen, Emina Torlak, and Xi Wang. 2020. Specification and verification in the field: Applying formal methods to BPF just-in-time compilers in the linux kernel. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 41--61.
[28]
Benjamin Curt Nilsen. 2020. Fuzzing the Berkeley Packet Filter. University of California, Davis.
[29]
Hui Peng, Zhihao Yao, Ardalan Amiri Sani, Dave Jing Tian, and Mathias Payer. 2023. GLeeFuzz: Fuzzing WebGL Through Error Message Guided Mutation. In 32nd USENIX Security Symposium (USENIX Security 23). 1883--1899.
[30]
IO Visor Project. 2019. eBPF fuzzing framework based on libfuzzer and clang sanitizer. https://github.com/iovisor/bpf-fuzzer
[31]
IO Visor Project. 2024. uBPF issue list. https://github.com/iovisor/ubpf/issues
[32]
IO Visor Project. 2024. Userspace eBPF VM. https://github.com/iovisor/ubpf
[33]
qmonnet. 2024. rBPF issues list. https://github.com/qmonnet/rbpf/issues
[34]
Qmonnet. 2024. Rust Virtual Machine and Jit Compiler for eBPF programs. https://github.com/qmonnet/rbpf
[35]
Simon Scannell. 2021. Fuzzing for eBPF JIT bugs in the Linux kernel. https://scannell.io/posts/ebpf-fuzzing
[36]
Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitriy Vyukov. 2012. AddressSanitizer: A fast address sanity checker. In 2012 USENIX annual technical conference (USENIX ATC 12). 309--318.
[37]
Solana. 2024. Solana rBPF issues list. https://github.com/solana-labs/rbpf/issues
[38]
Solana. 2024. Solana rBPF project fuzzer. https://github.com/solana-labs/rbpf/blob/f3758ecee89198433422f751beee7f0f52dbcd55/src/fuzz.rs
[39]
Solana. 2024. Solana Security Advisories. https://github.com/solana-labs/solana/security/advisories
[40]
Toshio Suganuma, Toshiaki Yasue, Motohiro Kawahito, Hideaki Komatsu, and Toshio Nakatani. 2001. A dynamic optimization framework for a Java just-in-time compiler. ACM SIGPLAN Notices, Vol. 36, 11 (2001), 180--195.
[41]
Hao Sun, Yiru Xu, Jianzhong Liu, Yuheng Shen, Nan Guan, and Yu Jiang. 2024. Finding Correctness Bugs in eBPF Verifier with Structured and Sanitized Program. (2024).
[42]
The Clang Team. 2024. Undefined Behavior Sanitizer. https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
[43]
V8 Team. 2023. V8's Fastest Optimizing JIT. https://v8.dev/blog/maglev
[44]
Vbpf. 2024. eBPF verifier based on abstract interpretation. https://github.com/vbpf/ebpf-verifier
[45]
vbpf. 2024. PREVAIL eBPF verifier issue list. https://github.com/vbpf/ebpf-verifier/issues
[46]
Marcos AM Vieira, Matheus S Castanho, Racyus DG Pacífico, Elerson RS Santos, Eduardo PM Câmara Júnior, and Luiz FM Vieira. 2020. Fast packet processing with ebpf and xdp: Concepts, code, challenges, and applications. ACM Computing Surveys (CSUR), Vol. 53, 1 (2020), 1--36.
[47]
Harishankar Vishwanathan, Matan Shachnai, Srinivas Narayana, and Santosh Nagarakatte. 2022. Sound, precise, and fast abstract interpretation with tristate numbers. In 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). IEEE, 254--265.
[48]
Junjie Wang, Zhiyi Zhang, Shuang Liu, Xiaoning Du, and Junjie Chen. 2023. FuzzJIT:Oracle-Enhanced Fuzzing for JavaScript Engine JIT Compiler. In 32nd USENIX Security Symposium (USENIX Security 23). 1865--1882.
[49]
Anatoly Yakovenko. 2018. Solana: A new architecture for a high performance blockchain v0. 8.13. Whitepaper (2018).
[50]
Rui Yang and Marios Kogias. 2023. HEELS: A Host-Enabled eBPF-Based Load Balancing Scheme. In Proceedings of the 1st Workshop on eBPF and Kernel Extensions. 77--83.
[51]
Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and understanding bugs in C compilers. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation. 283--294.
[52]
Mingwei Zheng, Qingkai Shi, Xuwei Liu, Xiangzhe Xu, Le Yu, Congyu Liu, Guannan Wei, and Xiangyu Zhang. 2024. ParDiff: Practical Static Differential Analysis of Network Protocol Parsers. Proceedings of the ACM on Programming Languages, Vol. 8, OOPSLA1 (2024), 1208--1234.
[53]
Yuhong Zhong, Haoyu Li, Yu Jian Wu, Ioannis Zarkadas, Jeffrey Tao, Evan Mesterhazy, Michael Makris, Junfeng Yang, Amy Tai, Ryan Stutsman, and Asaf Cidon. 2022. XRP: In-Kernel Storage Functions with eBPF. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). USENIX Association, Carlsbad, CA, 375--393. https://www.usenix.org/conference/osdi22/presentation/zhong
[54]
Shiyao Zhou, Muhui Jiang, Weimin Chen, Hao Zhou, Haoyu Wang, and Xiapu Luo. 2023. WADIFF: A Differential Testing Framework for WebAssembly Runtimes. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 939--950.
Index Terms
- Toss a Fault to BpfChecker: Revealing Implementation Flaws for eBPF runtimes with Differential Fuzzing
Recommendations
BRF: Fuzzing the eBPF Runtime
The eBPF technology in the Linux kernel has been widely adopted for different applications, such as networking, tracing, and security, thanks to the programmability it provides. By allowing user-supplied eBPF programs to be executed directly in the ...
Understanding Performance of eBPF Maps
eBPF '24: Proceedings of the ACM SIGCOMM 2024 Workshop on eBPF and Kernel ExtensionsThe Linux community has witnessed the rapid development of eBPF technology that allows users to load custom programs into the Linux kernel to extend its capabilities. A key feature that makes eBPF powerful is eBPF maps, which provide data storage and ...
On Integrating eBPF into Pluginized Protocols
eBPF is a popular technology originating from the Linux kernel that enables safely running user-provided programs in a kernel-context. This technology opened the door for efficient programming in the operating system, especially in its network stack. ...
Comments
Information & Contributors
Information
Published In
December 2024
5188 pages
ISBN:9798400706363
DOI:10.1145/3658644
- General Chairs:
- Bo Luo,
- Xiaojing Liao,
- Jun Xu,
- Program Chairs:
- Engin Kirda,
- David Lie
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 December 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- National Key R&D Program of China
Conference
CCS '24
Sponsor:
CCS '24: ACM SIGSAC Conference on Computer and Communications Security
October 14 - 18, 2024
UT, Salt Lake City, USA
Acceptance Rates
Overall Acceptance Rate 1,261 of 6,999 submissions, 18%
Upcoming Conference
CCS '25
- Sponsor:
- sigsac
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 187Total Downloads
- Downloads (Last 12 months)187
- Downloads (Last 6 weeks)120
Reflects downloads up to 27 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in