Data integration (DI) has been an area for intensive research for decades, which resulted in a few acknowledged reference architectures. The architectures can be categorized as supporting: (1) virtual integration (federated and mediated), (2) physical integration (data warehouse), and (3) hybrid (data lake, data lakehouse, data mesh). Regardless of their specific type, all these architectures rely on a complex integration layer. The layer is implemented by a sophisticated software, for designing, orchestrating, and running the so-called DI processes. On the one hand, in all business domains, large volumes of highly heterogeneous data are produced, e.g., medical systems, smart cities, smart agriculture, which require further advancements in the data integration technologies. On the other hand, the widespread adoption of artificial intelligence (AI) solutions is now extending towards DI, offering alternative solutions, opening new research paths, and generating new open problems. In this talk, I will share my perspective on the application and potential of AI solutions for selected DI problems. I will also highlight still unresolved issues within the field of DI. The talk will be structured into three main parts: (1) an overview of data integration architectures, (2) selected AI techniques for DI (like data wrangling, data quality, schema matching, optimization of systems, and code generation), and (3) still open problems in DI. The findings presented in the talk are based on my experience in running research and development DI projects for various business entities. It offers a concise overview of common DI challenges and potential solutions, serving as a quick-start guide for further exploration.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Data Integration in the AI Era: Research Trends and Still Open Issues

  • Robert Wrembel

摘要

Data integration (DI) has been an area for intensive research for decades, which resulted in a few acknowledged reference architectures. The architectures can be categorized as supporting: (1) virtual integration (federated and mediated), (2) physical integration (data warehouse), and (3) hybrid (data lake, data lakehouse, data mesh). Regardless of their specific type, all these architectures rely on a complex integration layer. The layer is implemented by a sophisticated software, for designing, orchestrating, and running the so-called DI processes. On the one hand, in all business domains, large volumes of highly heterogeneous data are produced, e.g., medical systems, smart cities, smart agriculture, which require further advancements in the data integration technologies. On the other hand, the widespread adoption of artificial intelligence (AI) solutions is now extending towards DI, offering alternative solutions, opening new research paths, and generating new open problems. In this talk, I will share my perspective on the application and potential of AI solutions for selected DI problems. I will also highlight still unresolved issues within the field of DI. The talk will be structured into three main parts: (1) an overview of data integration architectures, (2) selected AI techniques for DI (like data wrangling, data quality, schema matching, optimization of systems, and code generation), and (3) still open problems in DI. The findings presented in the talk are based on my experience in running research and development DI projects for various business entities. It offers a concise overview of common DI challenges and potential solutions, serving as a quick-start guide for further exploration.