MS-ETL: An Architecture for the Multiple Data Source Extraction, Transformation, and Load Applied to Solar Flares Data

Enabling the knowledge discovery in Series of Satellite Images is a challenging task. Spatiotemporal and non-traditional data constitute this domain, i.e., images and semi-structured textual data. Although there is a massive amount of applicability for this knowledge, we still face a lack of work that addresses the challenges. The gap is even worse when working with the Series of Solar Satellite Images (SSSI). Thus aiming to enable the extraction of knowledge in SSSI, we proposed a new Extraction, Transformation, and Load (ETL) architecture called Multiple data-source Solar ETL (MS-ETL). MS-ETL extracts SSSI data from multiples source and joins it in a single source of truth, enabling knowledge extraction: The solar images are extracted from a data source that is a different SSSI textual data source. After the extraction, MS-ETL transforms the textual data into structured data and loads it to a database allowing simple out-of-the-box analysis. In the end, we obtain a single source of truth for SSSI of almost two solar cycles in a reasonable amount of time. Now that information is available and can be used to apply advanced machine learning techniques. Notably, the proposed approach collects well-suited data for input in deep learning techniques.