RecTS: A Temporal-Aware Memory System Optimization for Training Deep Learning Recommendation Models

Deep learning recommendation models (DLRM) are crucial for various services, which makes DLRM training a critical workload for data centers. To meet the memory demands of DLRM workloads, the adoption of SSD has become necessary for large-scale models. However, the irregular access patterns of embedding tables in DLRM training result in poor spatial locality and low cache hit rates in the memory system. Furthermore, the mismatched access granularity between DRAM and SSD leads to write amplification and reduced SSD lifespan. To address these challenges, we propose RecTS: a temporal-aware memory system optimization for training deep learning recommendation models. The system uses an innovative replacement policy and vector grouping to achieve a high cache hit rate and improve spatial locality, which in turn reduces write amplification. These mechanisms are designed based on the access pattern of DLRM. During DLRM training, the access pattern of embedding tables is spatially irregular but predetermined. Leveraging these a priori access patterns, the system evicts the vectors and dynamically gathers the evicted embedding vectors with the closer next access timing into an I/O chunk to enhance spatial locality. This improvement makes the system more effective for DLRM workloads. We conducted experiments on both CPU-only and CPU-GPU architectures, where the difference lies in whether the neural network computation (MLP) is computed on the CPU or GPU. Overall, the replacement policy and vector grouping increase the cache hit rate from 40-85% to over 99% across all experiment settings, speeding up the training time by 1.30-5.55 times. Moreover, the SSD write volume was reduced by 3.13-7.30 times, significantly enhancing SSD lifespan.