Influence of Microservice Design Patterns for Data Science Workflows

Due to several advantages like scalability or fast development cycles, microservice architectures could also support the development and deployment of data science workflows. In recent literature, many patterns for the development of microservice architectures have evolved mainly for transaction-oriented applications. This paper investigates suitable design patterns for data science workflows. For this, we will implement a data science workflow using a microservice architecture, implemented with two different patterns, the orchestrator and choreography patterns, and with synchronous and asynchronous communication styles. Experiments are conducted to compare these architecture patterns with workloads of volume and velocity criteria. Orchestrator pattern performs best and could be used for inference, while choreography pattern could be used for training of machine learning models due to asynchronous communication. Our paper can provide practical support to software architects in implementing data science workflows using appropriate microservice design patterns. Also, we have now combined microservices pattern with data science workflows.