This chapter outlines a digitalizationDigitalization pattern for data integrationData integration in complex environmentsEnvironment. The idea is applying the extract-transform-load (ETL) type of transformation to heterogeneous data warehousesData Warehouse (DWH) and dataData handling toolsTool. In these large-scale warehouses, dataData size, dynamics, heterogeneity, and diversity, taken together, constitute a dramatic crisisCrisis trigger. To facilitate the above-mentioned compatibility, a layer-based architectural pattern is suggested. To provide the required level of flexibility, this common pattern features metadataMetadata modeling facilities and management instruments. This serviceService-oriented methodologyMethodology also includes microservicesMicroservice, publish–subscribe event handling, and open-source APIAPI to facilitate loosely coupled module interactionInteraction, which is essential for crisisCrisis-resilient dataData management. DataData orchestration and choreography are suggested as possiblePossible ways to conquer heterogeneity. The intelligent toolkit used includes Kubernetes, Apache Airflow, and a machine learningMachine Learning (ML)-based recommending mechanism to support versatile dataData and social network integration.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Applying Microservices: Data Integration and Graphical Interface Testing

  • Sergey V. Zykov

摘要

This chapter outlines a digitalizationDigitalization pattern for data integrationData integration in complex environmentsEnvironment. The idea is applying the extract-transform-load (ETL) type of transformation to heterogeneous data warehousesData Warehouse (DWH) and dataData handling toolsTool. In these large-scale warehouses, dataData size, dynamics, heterogeneity, and diversity, taken together, constitute a dramatic crisisCrisis trigger. To facilitate the above-mentioned compatibility, a layer-based architectural pattern is suggested. To provide the required level of flexibility, this common pattern features metadataMetadata modeling facilities and management instruments. This serviceService-oriented methodologyMethodology also includes microservicesMicroservice, publish–subscribe event handling, and open-source APIAPI to facilitate loosely coupled module interactionInteraction, which is essential for crisisCrisis-resilient dataData management. DataData orchestration and choreography are suggested as possiblePossible ways to conquer heterogeneity. The intelligent toolkit used includes Kubernetes, Apache Airflow, and a machine learningMachine Learning (ML)-based recommending mechanism to support versatile dataData and social network integration.