MetaSys: A Practical Open-source Metadata Management System to Implement and Evaluate Cross-layer Optimizations

Similar to prior systems for taint-tracking, security, and performance optimization, MetaSys implements tagged memory-based [53, 68, 173, 182] metadata management. MetaSys associates metadata with memory address ranges of arbitrary sizes by tagging each memory address with an 8-bit (configurable) ID or tag. Each tag is a unique pointer to metadata that describes the data at the memory address. Hardware optimizations (e.g., in the cache, memory controller, or core) can query for the tag associated with any memory address and the metadata associated with the tag.

The mapping between each memory address and the corresponding ID is saved in a table in memory referred to as Metadata Mapping Table (MMT): ❹ in Figure 1. This table is allocated by the OS for each process and is saved in memory. In MetaSys (similar to XMem [164] and Cheri [174]), we tag physical addresses. As a result, any virtual address has to be translated before indexing the MMT to retrieve the tag ID. To enable fast retrieval of IDs, we implement a cache for the MMT in hardware that stores frequently accessed mappings, referred to as the Metadata Mapping Cache (MMC) ❺. MMC misses lead to memory accesses to retrieve mappings from the MMT in memory.

MetaSys can be configured to tag memory at flexible granularities. In Section 9.1, we evaluate the performance impact of the tagging granularity. The size of the MMT depends on the tagging granularity. For a 512 B mapping granularity, the MMT requires 0.2% of physical memory (16 MB in a 8 GB system). The MMC holds 128 entries, where each entry stores a physical-address-to-tag mapping, and is 608 B in size (8 bit entry and 30 bit tag).

We implement dedicated mapping tables for tag IDs rather than use the page table or TLBs for the following reasons: First, doing so obviates the need to modify the latency-critical address translation structures. Second, MetaSys associates physical addresses with Tag IDs rather than virtual addresses (to enable the memory controller and LLCs to look up metadata). Thus, a page table or TLB cannot be directly used to save Tag IDs as they are indexed with virtual addresses.

The actual metadata associated with any ID is saved in special SRAM caches that are private to each hardware component or optimization. For example, the prefetcher would separately save access pattern information, while a hardware bounds checker would privately save data structure boundary information. We refer to these stores as Private Metadata Tables (PMTs) ❻. The PMTs are saved near each component (private to each component) and are loaded/updated by MetaSys. The metadata (e.g., locality/“hotnes”) is encoded such that it can be directly interpreted by the component, e.g., a prefetcher.

Communicating application information with MetaSys requires (i) associating memory address ranges with a tag or ID of configurable size (8 bits by default) and (ii) associating each ID with the relevant metadata. The metadata could include program properties that describe the memory range, such as data locality/reuse, access patterns, read-write characteristics, data “hotness,” and data types/layouts. We use two operators (described below) that can be called in programs to dynamically communicate metadata.

To associate memory address ranges with an ID, we provide the MAP/UNMAP interface ❼ (similar to XMem [164]). MAP and UNMAP are implemented as new RISC-V instructions that are interpreted by the Mapping Management Unit (MMU) to map a range of memory addresses (from a given virtual address up to a certain length) to the provided ID. These mappings are saved by the MMU in the MMT. We also implement 2D and 3D versions of MAP to efficiently map two-/three-dimensional address ranges in a multi-dimensional data structure with a single instruction.

To associate each ID with metadata, we provide the CREATE interface. CREATE ❽ takes three inputs from the application: the tag ID, the 8-bit ID for the hardware component (i.e., prefetcher, bounds checker, etc., called Module ID), and 512 B of metadata. CREATE directly populates the PMT of the appropriate hardware component with 512 B of (or less) metadata. Each PMT (private to the optimization client) has 256 entries assuming 8-bit tag IDs. The CREATE operator overwrites the metadata at the entry indexed by the tag ID at the PMT specified by the module ID. All CREATE and MAP instructions are associated with the next load/store instruction in program order to avoid inaccuracies due to out-of-order execution. In other words, an implicit dependence is created in hardware between these instructions and the next load/store, and they are committed together. This enables associating information with the next load/store and not just the memory region associated with it, e.g., in the bounds checking use case described in Section 6.1.

Table 1 lists the new instructions along with their arguments.

Each optimization component is triggered by a hardware event ❾ (e.g., a cache miss). A component then retrieves the physical address corresponding to the virtual address associated with the event (e.g., the virtual address that misses in the cache) from the TLB ❿ (in case of L1 optimizations) and queries the MMC with the physical address to retrieve the associated tag ID. On a miss in the MMC, the mapping is retrieved from the MMT in memory. The optimization client uses the retrieved tag ID to obtain the appropriate metadata from the PMT. The optimization client is designed to flexibly implement a wide range of use cases and can be designed based on the optimization at hand. For example, the optimization client used to build the prefetcher use case in Section 5 has interfaces to the prefetcher, caches, memory controller, and TLBs to make implementing optimizations easier. Each client has a static ID (clientID) and a PMT that is updated by the CREATE operator.

We add OS support for metadata management in the RISC-V proxy kernel [128], which can be booted on our Rocket RISC-V prototype: First, we add support to manage the MMT in memory, where the OS allocates the MMT in the physical address space and communicates the pointer to the MAP hardware support. Second, we add support to flush the PMTs during a context switch (similar to how the TLB is flushed). Third, if the OS changes the virtual to physical address mapping of a page, then to ensure consistency of the metadata, the MMT is updated by the OS to reflect the correct physical-address-to-tag-ID mapping and the corresponding MMC entries are invalidated. We modify the page allocation mechanism in the OS to do this. In addition, we also provide support to implement optimizations performed by the OS or with OS cooperation. To do so, MetaSys enables trapping into the OS to perform customized checks or optimizations (e.g., protection checks or altering virtual-to-physical mappings) based on specific hardware trigger events (using interrupt routines). We describe one such use case in Section 6.

MetaSys can be flexibly extended to multicore processors. Metadata is maintained at a process-level, therefore, threads within the same process cannot have different metadata for the same data structure. The MMC is a per-core structure, while the Private Metadata Tables (PMTs) are per-component structures (e.g., at the memory controller, LLC, prefetcher). The two dynamic operators (CREATE and MAP) may cause challenges in coherence and consistency of metadata in multicore systems. CREATE directly updates metadata associated with the per-process tag ID, which is saved at the per-component PMTs. The PMTs are shared by all cores when the optimization component is also shared (and thus any updates by CREATE are automatically coherent). The PMTs for private components (e.g., L1 cache) are not coherent and can only be updated by the corresponding thread. MAP updates the mapping in the MMC, which is private to each core. To ensure coherence of the MMC mappings, a MAP update invalidates the corresponding MMC entry (if present) in other MMCs by broadcasting updates with a snoopy protocol. If the use case requires consistency of the metadata, i.e., ordering between a CREATE/MAP instruction and when it is visible to other cores, then barriers and fence instructions are used to enforce any required ordering between threads for updates to metadata.

MetaSys supports three modes: (i) Force stall, where the instruction triggering a metadata lookup cannot commit until the optimization completes (e.g., for security use cases); (ii) No stall, where metadata lookups do not stall the core but are always resolved (e.g., for page placement, cache replacement); and (iii) Best effort, where lookups may be dropped to minimize performance overheads (e.g., for prefetcher training).

We develop a software library that can be included in user programs to facilitate the use of MetaSys primitives CREATE and MAP (Table 2). The library exposes three functions: (i) CREATE populates an entry indexed by the tag ID (TagID) in the PMT of a hardware optimization client (ClientID) with the corresponding metadata; (ii) MAP updates the MMT by assigning tag IDs to memory addresses of the range (start, end); (iii) UNMAP resets the tag IDs of the corresponding address range in the MMT. While the operators can be directly used via the provided software library, their use can be simplified by using wrapper libraries that abstract away the need to directly manage tag IDs and their mappings.

MetaSys implements a tagged-memory-based system with a metadata cache similar to XMem [164]. MetaSys however has three major benefits over XMem. First, MetaSys enables communicating metadata at runtime using a more powerful CREATE operator that is implemented as a new instruction. In XMem, metadata is communicated only statically at compile time (CREATE is hence a compiler pragma). MetaSys thus enables a wider set of optimizations including fine-grained memory safety, protection, prefetching, and so on, and enables communicating metadata that is dependent on program input and metadata that can be accurately known only at runtime (e.g., access patterns, data “hotness,” etc.). MetaSys was designed to efficiently handle these dynamic metadata updates. Second, the dynamic and more expressive CREATE operator obviates the need for additional interfaces (ACTIVATE/DEACTIVATE) to track the validity of statically communicated metadata. This enables a more streamlined metadata system in MetaSys with fewer new instructions, tables, and lookups. Third, MetaSys allows the application programmer to directly select which cross-layer optimization to enable/disable and communicate metadata to, via the CREATE operator. XMem, however, does not allow control of hardware optimizations from the application. Table 3 summarizes the MetaSys operators and compares to the corresponding operators in XMem. Of the three MetaSys use cases we evaluate in this article, only return address protection (Section 6.2) can be implemented with XMem.

We build a full system prototype of MetaSys on an FPGA with the Rocket Chip RISC-V system [10] and add the necessary support in the compiler, libraries, OS, ISA, and hardware. The modularized MetaSys components can also be ported to other RISC-V cores. We used the RoCC accelerator [10] in the Rocket chip to implement the metadata management system. RoCC is a customizable module that enables interfacing with the core and memory. The hardware support implemented in ROCC comprises (i) the control logic to handle MAPs and CREATEs, (ii) control logic to perform metadata lookups by components that implement optimizations, and (iii) the memory for metadata caches (MMC and PMTs). We extended the RISC-V ISA with eight instructions (six for MAP/CREATE and two for OS operations). To implement all the hardware modules of MetaSys, we modified/added 1,781 lines of Chisel code in the Rocket Chip. As we demonstrate later, since the MetaSys hardware modules can be flexibly reused across multiple hardware-software optimizations, the techniques in our use cases only required 87–103 additional lines of Chisel code. The full MetaSys infrastructure is open-sourced [57] including the Chisel code for the MetaSys hardware support, the RISC-V OS with the required modification, and the software libraries to expose the MetaSys primitives.

To implement a new hardware technique with the baseline MetaSys code, we provide a flexible module (❶ in Figure 2) with a PMT and interfaces to the metadata lookup unit, to the core (to receive triggers), and interfaces to the cache controller. The interface to the lookup unit ❷ provides dynamic access to the metadata communicated by the CREATE and MAP operators. The interfaces to the core ❸ and the memory system ❹ can be used as trigger events for optimization and lookups (e.g., a cache miss). The different components within the MetaSys logic itself (i.e., the metadata caches, logic to access the Metadata Mapping Table in memory, and the lookup logic) can be flexibly reconfigured.

MetaSys relies heavily on function calls/libraries that abstract away low-level details that call the MetaSys instructions even in C/C++. With managed and dynamically typed languages, the metadata associated with data structures/objects would be provided by the user with additional class/object member functions. The metadata could also be directly embedded within object/class definitions (e.g., a list or map in Python would by definition have certain access properties). Other properties (e.g., data types) would be provided by the interpreter (in the case of dynamically typed languages) and the mapping/remapping calls to memory addresses would be handled by the runtime during memory (de)allocation.

In comparison to specialized cross-layer solutions, MetaSys offers the following benefits: (i) Generality: toward implementing a large number of use cases, including more complex use cases such as specialized prefetching (Section 5), which amortizes the overall hardware cost; (ii) Flexibility and versatility in the implemented instructions: A challenge with specialized cross-layer solutions is the need to add new instructions that create challenges in forward/backward compatibility and also require changes across the stack for each new optimization. With MetaSys, the instructions are designed to be agnostic to the optimization and only require a one-off change to the hardware-software interface; (iii) Infrastructure for evaluation: MetaSys can be used to implement many specialized cross-layer techniques in real hardware, which would otherwise be a challenging programming task (as demonstrated in Sections 5 and 6). In Sections 5 and 6, we evaluate MetaSys’s ability to implement several cross-layer techniques.

Vertex-centric graph analytics typically involves first traversing a work list containing vertices to be visited (❶ in Figure 3, left). For each vertex, the application accesses the vertex list ❷ to retrieve the neighboring vertex IDs from the edge list ❸. To perform computation on the graph, the application then operates on the properties of these neighboring vertices (retrieved from the property list ❹). Graph processing thus involves a series of memory accesses that depend on the contents of the work, vertex and edge lists.

In this use case, we design a prefetcher that can interpret the contents of each of the above data structures and appropriately compute the next data-dependent memory address to prefetch. To capture the required application information for each data structure, we use MetaSys’s CREATE interface to communicate the following metadata: (i) base address of the data structure that is indexed using the current data structure’s contents (64 bits); (ii) base address of the current data structure (64 bits); (iii) data type (32 bits) and size (32 bits) to determine the index of the next access; and (iv) the prefetching stride (6 bits). MAP then associates the address range of each data structure with the appropriate tag.

Listing 1 shows a detailed end-to-end example of how metadata is created in the application (BFS), how metadata tags are associated with the data structures of BFS, and how the prefetcher operates. Lines 2-10 (incorporated into the code of the BFS application) use the MetaSys software libraries to create metadata (with CREATE) and associate it with the corresponding data structures (using MAP). CREATE saves the metadata in the PMT and MAP updates the MMT. Lines 13-21 (incorporated into the hardware optimization client responsible for prefetching) describe the algorithm behind the MetaSys-based prefetcher. The prefetcher is implemented with an optimization client (ClientID = 0). The prefetcher essentially: (i) snoops every memory request from the core and retrieves the associated tag ID using MetaSys; (ii) queries the PMT to retrieve the communicated metadata (listed above); and (iii) uses the metadata to identify dependencies between the data structures of the application.

We describe a detailed walkthrough of how the prefetcher operates during the execution of the BFS application using Figure 3 and Listing 1. In Figure 3 (left), when the prefetcher snoops a memory request that targets the work list at index 0, it looks ahead (depending on the prefetching stride) to retrieve the contents of the work list at index 1. At this point, it also prefetches the contents of the vertex, edge, and property lists based on the computed index at each level. In graph applications where the work list is ordered, the prefetcher is configured to simply stream through the contents of the vertex and edge lists to prefetch the data dependent memory locations in the property list. The \(snoop\_mem\_request(address)\) (Line 13) function is executed for each request sent by the core to the memory hierarchy. For every memory request, the prefetcher accesses the MMC using the address to receive the tag ID (using MetaSys’s lookup functionality). Next, it indexes the PMT using the tag ID to retrieve the metadata associated with the memory request. Using the metadata, the prefetcher determines if the request comes from one of the data structures of the application (Line 17). In this case, the prefetcher first prefetches ahead (Line 18) according to the stride and waits until it receives the value of the prefetched request (Line 19). Using the value, it calculates the address of the data-dependent data structure (e.g., value of WorkList used as an index for VertexList) and looks up the metadata for the newly formed address. The same procedure happens until no further data-dependency is found (Line 16).

The prefetcher can be flexibly configured (by associating metadata to data structures, Lines 2–10 in Listing 1) by the user based on the specific properties associated with any data structure, algorithm, and the desired aggressiveness of prefetcher.

We evaluate the MetaSys-based prefetcher using eight graph analytics workloads from the Ligra framework [145] using the Rocket Chip prototype of MetaSys with the system parameters listed in Table 4. We evaluate three configurations: (i) the baseline system with a hardware stride prefetcher [55]; (ii) GraphPref, a customized hardware prefetcher that implements the same idea described above without the generalized MetaSys support (similar to prior work [5, 157]); and (iii) the MetaSys-based graph prefetcher. In the case of GraphPref, all the required metadata (e.g., base and bound addresses, stride) are directly provided to the prefetcher using specialized instructions. Thus, GraphPref is able to access metadata at low latency and does not access the memory hierarchy. The prefetcher works in the same way as the MetaSys-based prefetcher, however, in the case of MetaSys, the general CREATE/MAP instructions are used to communicate information and the metadata lookups access the MMC (which may lead to additional memory accesses when there is an MMC miss). Figure 3 (right) depicts the corresponding speedups, normalized to the baseline. We observe that the MetaSys graph prefetcher improves performance by 11.2% on average (up to 14.3%) over the baseline by accurately prefetching data-dependent memory accesses. It also significantly outperforms the stride prefetcher, which is unable to capture the irregular access patterns in graph workloads. Compared to GraphPref, the MetaSys-based prefetcher performs almost as well: within 0.2% on average (within 0.8% for BFS). The additional overheads of MetaSys come from the MMC misses and the larger number of instructions used. In terms of area, MetaSys requires 17 KB of SRAM (1 KB for the MMC and 16 KB for the Private Metadata Table) compared to the custom hardware prefetcher, which requires 8 KB of SRAM for the metadata. The custom prefetcher requires two additional instructions and additional logic to perform metadata lookups and create/update metadata. We found the area complexity to be slightly less for the custom solution as the SRAM requirements are lower ( \(\sim\) 0.01% for custom hardware versus \(\sim\) 0.02% for MetaSys, compared to a 22 nm Intel CPU Core [144]). However, MetaSys ’s overhead can be amortized over multiple use cases, whereas a custom solution is specific to a single use case.

We conclude that MetaSys can be used to flexibly implement and evaluate hardware-software cooperative techniques for prefetching by leveraging MetaSys’s metadata support and interfaces, incurring only small overheads from MetaSys’s general metadata management.

Unmanaged languages such as C/C++ provide great flexibility in memory management but an important challenge with these languages is memory safety. The pointer casting and pointer arithmetic supported by these languages allow buffer overflows and potentially hazardous writes to arbitrary memory locations. Prior work has demonstrated a range of software approaches [11, 34, 46, 50, 60, 67, 79, 100, 111, 112, 114, 119, 133, 173, 177, 180] to increase memory safety in the form of static or dynamic checks, such as CCured [112], Cyclone [67], and Softbound [111]. These approaches are known to incur significant runtime overheads in performing numerous checks in software [156]. Hardware-based approaches offer a promising opportunity to alleviate these overheads. Prior work [42, 43, 84, 98, 109, 110, 161, 174], including HardBound [43], ShaktiT [98], and Cheri [174] investigate enabling hardware-software cooperative bounds checking. These approaches require architectures that are entirely specialized for bounds checking [43, 84, 98] or more heavyweight metadata management systems tailored for memory security and protection [42, 109, 110, 161, 174]. In this section, we demonstrate how MetaSys can be used to implement hardware-based bounds checking at low overhead using a lightweight and general metadata system.

The program’s call stack is a known source of many security vulnerabilities in low-level, memory-unsafe languages such as C/C++. For example, the control flow in the program can be hijacked by overwriting the return addresses saved in the stack [8, 36]. Existing defenses such as ExecShield [160] and stack canaries [36] do not protect against sophisticated attack techniques [24, 28, 158]. Stack canary protection [36] is a software check that involves writing an additional randomly generated value in the stack and a duplicate is saved separately in memory. The stack canary checks the randomly generated value in the stack against its duplicate to detect stack overwriting before a function returns to the return address saved in the stack. Protecting return addresses with more powerful software checks [1, 16, 83, 97, 101] incurs significant runtime overheads and are hence difficult to use in practice [39, 156]. Prior work has proposed a range of hardware techniques [45, 63, 86, 129, 174, 186] to enable return address protection more efficiently. These approaches either require dedicated hardware support for stack protection (e.g., RAGuard [186], PAC-it-up [86], CET [63]) or more heavy-weight metadata systems for memory protection (e.g., SDMP [129], Cheri [174], PUMP [45]). In this section, we implement and evaluate return address protection with MetaSys’s lightweight metadata support and cross-layer interfaces.

We perform our characterization using the Polybench [124] and Ligra [145] benchmark suites along with a set of microbenchmarks (available on Github [57]). Polybench contains building block kernels frequently used in linear algebra, scientific computation, and machine learning. Ligra contains widely used graph analytics workloads. The microbenchmarks are designed to intensely stress the MetaSys system and identify worst-case overheads.

Section 4 describes the parameters of our baseline system. We summarize our key findings in Section 9.3. In all evaluations in this section, since we aim to characterize the overheads of the system itself, we do not implement any cross-layer optimization that improves performance. We simply implement lookups to the metadata system that an optimization could potentially make. Since our goal is to stress the system and understand the worst-case overheads, we perform metadata lookups for every memory access. In typical use cases, the lookup triggers would be much less frequent than for every memory access, e.g., lookups on only cache misses for prefetching or only stores for return address protection.

We synthesized the baseline MetaSys system using the Synopsys DC [155] at 22 nm process technology to estimate the area overhead. MetaSys incurs small area overhead: 0.03 \(\text{ mm}^2\) (0.02% of a 22 nm Intel Ivy Bridge CPU Core [144]).