A Microservice System Architecture for Receiving ETL System Patterns
摘要
The use of a service-oriented approach in ETL (Extract-Transform-Load) systems implementation is potentially very beneficial, as it allows transforming traditional ETL tasks into services, having simple and well-structured functionalities that can be reused according to the needs of the tasks and the data they handle. Furthermore, service-based approaches simplify the work of designing and implementing any ETL system that needs to be customized, as it provides a well-established, flexible and scalable infrastructure, highly modular, allowing reusing and simplify restructuring existing services in the implementation of new ETL components. In pattern based ETL systems, the use of service-based architecture allows mapping in a very natural way ETL patterns as services and orchestrate them easily using native management services. In this paper, we present, describe and discuss how we made pattern-service mapping and implemented a service-based ETL system in a concrete application domain, highlighting how ETL patterns can be materialized using a microservice architecture.