The landscape of distributed Database Management Systems (DBMSs) has been fundamentally transformed by advancements in data center network technologies. Traditionally, these systems were designed to minimize network communication, which was perceived as a significant bottleneck. However, with the advent of high-speed networks and technologies such as Remote Direct Memory Access (RDMA), this perception is changing. RDMA allows direct memory access to remote machines at speeds comparable to local memory access, challenging the long-standing belief that network communication is inherently slow. This chapter explores how the evolution of data center networks, mainly through RDMA, has enabled new designs for disaggregated databases. Moreover, the chapter delves into programmable networks as a solution beyond RDMA. In particular, this chapter provides an overview of RDMA and programmable networks and introduces novel ideas for optimizing databases in disaggregated setups using RDMA. Furthermore, two approaches are highlighted that leverage programmable networks in addition to RDMA to improve the efficiency of disaggregated databases even further: P4DB, which leverages programmable data planes for in-network OLTP processing, and zero-sided RDMA, a network-driven data shuffling method that enhances communication efficiency between heterogeneous accelerators. By showcasing these advancements, we demonstrate how database systems can utilize specialized networking hardware to achieve unprecedented performance levels. The chapter concludes with a summary and discussion of future research directions in the field of data center network technologies and their impact on distributed DBMSs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Scalable Data Management on Next-Generation Data Center Networks

  • Matthias Jasny,
  • Tobias Ziegler,
  • Carsten Binnig

摘要

The landscape of distributed Database Management Systems (DBMSs) has been fundamentally transformed by advancements in data center network technologies. Traditionally, these systems were designed to minimize network communication, which was perceived as a significant bottleneck. However, with the advent of high-speed networks and technologies such as Remote Direct Memory Access (RDMA), this perception is changing. RDMA allows direct memory access to remote machines at speeds comparable to local memory access, challenging the long-standing belief that network communication is inherently slow. This chapter explores how the evolution of data center networks, mainly through RDMA, has enabled new designs for disaggregated databases. Moreover, the chapter delves into programmable networks as a solution beyond RDMA. In particular, this chapter provides an overview of RDMA and programmable networks and introduces novel ideas for optimizing databases in disaggregated setups using RDMA. Furthermore, two approaches are highlighted that leverage programmable networks in addition to RDMA to improve the efficiency of disaggregated databases even further: P4DB, which leverages programmable data planes for in-network OLTP processing, and zero-sided RDMA, a network-driven data shuffling method that enhances communication efficiency between heterogeneous accelerators. By showcasing these advancements, we demonstrate how database systems can utilize specialized networking hardware to achieve unprecedented performance levels. The chapter concludes with a summary and discussion of future research directions in the field of data center network technologies and their impact on distributed DBMSs.