A Temporal Metadata Management Approach for Data Lakes

A data lake stores heterogeneous big data, in their native format, without any predefined schema, while providing supports for querying and analyzing such big data. Metadata are necessary for describing the big data stored in the data lake, and metadata management and querying are among the most important functionalities of a data lake management system. However, although metadata are temporal by their nature, existing metadata models for data lakes do not provide support for managing the evolution over time of metadata; the conventional metadata versioning that is supported by some of these models does not timestamp data versions and does manage these versions according to the rules and operations already defined in the temporal database field for the management of time-varying data. For these reasons, we propose in this paper a temporal metadata management approach for data lakes. This approach is based on a temporal metadata model for data lakes, named T-goldMEDAL, defined as a temporal extension of the conventional metadata model goldMEDAL; this latter has been chosen since it is the most generic/abstract and flexible model among those published in the literature of data lakes. Moreover, to make our model useful, we complete our approach with the proposal of a temporal query language, named QL4-T-goldMEDAL, for querying temporal metadata in a T-goldMEDAL data lake.