PESC: A Per System-Call Stack Canary Design for Linux Kernel

Stack canary is the most widely deployed defense technique against stack buffer overflow attacks. However, since its proposition, the design of stack canary has very few improvements during the past 20 years, making it vulnerable to new and sophisticated attacks. For example, the ARM64 Linux kernel is still adopting the same design with StackGuard, using one global canary for the whole kernel. The x86_64 Linux kernel leverages a better design by using a per-task canary for different threads. Unfortunately, both of them are vulnerable to kernel memory leaks. Using the memory leak bugs or hardware side-channel attacks, e.g., Meltdown or Spectre, attackers can easily peek the kernel stack canary value, thus bypassing the protection. To address this issue, we proposed a fine-grained design of the kernel stack canary named PESC, standing for Per-System-Call Canary, which changes the kernel canary value on the system call basis. With PESC, attackers cannot accumulate any knowledge of prior canary across multiple system calls. In other words, PESC is resilient to memory leaks. Our key observation is that before serving a system call, the kernel stack is empty and there are no residual canary values on the stack. As a result, we can directly change the canary value on system call entry without the burden of tracking and updating old canary values on the kernel stack. Moreover, to balance the performance as well as the security, we proposed two PESC designs: one relies on the performance monitor counter register, termed as PESC-PMC, while the other one uses the kernel random number generator, denoted as PESC-RNG. We implemented both PESC-PMC and PESC-RNG on the real-world hardware, using HiKey960 board for ARM64 and Intel i7-7700 for x86_64. The synthetic benchmark and SPEC CPU2006 experimental results show that the performance overhead introduced by PESC-PMC and PESC-RNG on the whole system is less than 1%.