Polyglot Persistence in Heterogeneous NoSQL Database Systems with Python
摘要
Modern applications often require functionalities that single types of database systems cannot adequately fulfil, prompting the use of multiple types of databases for effective data management. Polyglot persistence enables the integration of various data storage technologies within a single application, leveraging the strengths of each system. This approach, however, introduces complexities in design and implementation, increasing programming complexity and necessitating knowledge of multiple languages. This article presents a practical method for achieving application-coordinated polyglot persistence across key-value, column-oriented, and document-oriented NoSQL databases, specifically focusing on Redis, Cassandra, and MongoDB, using Python. Our proposed three-layer architecture enhances data management and analytics by utilizing Jupyter Notebooks for interactive applications and employing Python libraries for orchestration, integrating specialized NoSQL databases. We encountered specific challenges, such as memory overload in Cassandra during complex query operations, which highlights the limitations in scalability under certain conditions. Additionally, our findings indicate significant variations in query execution times, with MongoDB demonstrating the fastest responses and Cassandra showing delays when handling complex queries. Using the SSB from TPC-H with a dataset of 6,000,000 entries, we evaluated loading times and query execution performance. This study aims to serve as a comprehensive guide for researchers and industry professionals seeking to implement a unified interface across heterogeneous NoSQL engines, providing insights into both the potentials and hurdles of polyglot persistence.