Managing Very Large Datasets on Directly Attached NVMe Arrays
摘要
High-performance solid-state drives based on flash memory have replaced hard disks as the primary storage medium. Modern servers can host ten or more NVMe SSDs, and the aggregate bandwidth is approaching main memory bandwidth. In response to this evolution, this project evaluates the performance trade-offs posed by the abstraction layers and interfaces involved in I/O management. We experimentally slice through the software stack, exploring the limits of how efficient I/O can be done, and provide a discussion about durability. Finally, we discuss the implications of fast storage devices on the architecture and implementation of database storage engines. Overall, our work shows that exploiting the full potential of modern NVMe storage arrays is achievable but requires substantial engineering effort.