Applying Microservices: Data Integration and Graphical Interface Testing
摘要
This chapter outlines a digitalizationDigitalization pattern for data integrationData integration in complex environmentsEnvironment. The idea is applying the extract-transform-load (ETL) type of transformation to heterogeneous data warehousesData Warehouse (DWH) and dataData handling toolsTool. In these large-scale warehouses, dataData size, dynamics, heterogeneity, and diversity, taken together, constitute a dramatic crisisCrisis trigger. To facilitate the above-mentioned compatibility, a layer-based architectural pattern is suggested. To provide the required level of flexibility, this common pattern features metadataMetadata modeling facilities and management instruments. This serviceService-oriented methodologyMethodology also includes microservicesMicroservice, publish–subscribe event handling, and open-source APIAPI to facilitate loosely coupled module interactionInteraction, which is essential for crisisCrisis-resilient dataData management. DataData orchestration and choreography are suggested as possiblePossible ways to conquer heterogeneity. The intelligent toolkit used includes Kubernetes, Apache Airflow, and a machine learningMachine Learning (ML)-based recommending mechanism to support versatile dataData and social network integration.