The Impact of Data Ingestion Layer in an Improved Lambda Architecture

The explosion of connected objects and increasing Internet flows generate a variety of large amounts of data. More often the data generation speeds is difficult to control for traditional data analysis systems. That is, analyzing the generated data in real-time and in batch process modes appears necessary. The Lambda architecture has been proposed in the literature with a data ingestion layer for data collection, filtration, transformation and transfer to address this challenge. Despite the diversity of data and its high generation speed, the Lambda architecture must enable companies to process efficiently data. However, the Lambda architecture presents some difficulties, particularly in its implementation process, the separation of processing, and data transfer and synchronization. In this work, we propose a variant of the Lambda architecture which decouples the data ingestion layer from the processing layer, to facilitate its implementation. The simulation results show that there is value in implementing the new proposed architecture.