The use of a service-oriented approach in ETL (Extract-Transform-Load) systems implementation is potentially very beneficial, as it allows transforming traditional ETL tasks into services, having simple and well-structured functionalities that can be reused according to the needs of the tasks and the data they handle. Furthermore, service-based approaches simplify the work of designing and implementing any ETL system that needs to be customized, as it provides a well-established, flexible and scalable infrastructure, highly modular, allowing reusing and simplify restructuring existing services in the implementation of new ETL components. In pattern based ETL systems, the use of service-based architecture allows mapping in a very natural way ETL patterns as services and orchestrate them easily using native management services. In this paper, we present, describe and discuss how we made pattern-service mapping and implemented a service-based ETL system in a concrete application domain, highlighting how ETL patterns can be materialized using a microservice architecture.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Microservice System Architecture for Receiving ETL System Patterns

  • Rui Monteiro,
  • Bruno Oliveira,
  • Orlando Belo

摘要

The use of a service-oriented approach in ETL (Extract-Transform-Load) systems implementation is potentially very beneficial, as it allows transforming traditional ETL tasks into services, having simple and well-structured functionalities that can be reused according to the needs of the tasks and the data they handle. Furthermore, service-based approaches simplify the work of designing and implementing any ETL system that needs to be customized, as it provides a well-established, flexible and scalable infrastructure, highly modular, allowing reusing and simplify restructuring existing services in the implementation of new ETL components. In pattern based ETL systems, the use of service-based architecture allows mapping in a very natural way ETL patterns as services and orchestrate them easily using native management services. In this paper, we present, describe and discuss how we made pattern-service mapping and implemented a service-based ETL system in a concrete application domain, highlighting how ETL patterns can be materialized using a microservice architecture.