An E-Commerce Benchmark for Evaluating Performance Trade-Offs in Document Stores

In document stores, application developers are faced with complicated schema design trade-off decisions between embedding or referencing of document relationships. However, existing performance benchmark systems for document stores either work with simplistic and unrepresentative documents, or involve non-document store-specific workloads, and none of the existing benchmarks allows evaluating the performance impact of the fine-grained trade-offs between embedding and referencing in realistic application cases. In this paper, we present the design of a novel benchmark which consists of a data model and corresponding workload consisting of ten representative queries. The evaluated trade-offs are based on a study of two specific open source e-commerce systems.