Scholarlink

<p>In recent years, data generated from diverse sources has grown exponentially, giving rise to new challenges for processing and analysis under the umbrella of “big data". Although various methods and platforms have been proposed to tackle these issues, there is still a need for a conceptual guide that synthesizes state-of-the-art tools and techniques across the full machine learning (ML) workflow. In this survey, we consolidate research on big data infrastructures, distributed processing frameworks, and ML methods, mapping out an end-to-end conceptual framework that can serve as a reference architecture for end-to-end deployment. Specifically, we discuss key aspects of big data analytics (storage solutions, platforms like Hadoop and Spark, NoSQL databases), preprocessing approaches (dimensionality reduction, instance selection, noise handling), ML algorithms (supervised, unsupervised, and emerging deep learning paradigms), and deployment considerations (monitoring, continuous integration, and versioning). By gathering and integrating these elements, our survey provides a comprehensive overview of existing solutions, clarifies the design choices developers must consider, and identifies research gaps that remain to be addressed.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Practical big data techniques for end-to-end machine learning deployment: a comprehensive review

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年4月15日

查看全文
  • Ahmed El-Sayed,
  • Mohamed Abougabal,
  • Shaimaa Lazem

摘要

In recent years, data generated from diverse sources has grown exponentially, giving rise to new challenges for processing and analysis under the umbrella of “big data". Although various methods and platforms have been proposed to tackle these issues, there is still a need for a conceptual guide that synthesizes state-of-the-art tools and techniques across the full machine learning (ML) workflow. In this survey, we consolidate research on big data infrastructures, distributed processing frameworks, and ML methods, mapping out an end-to-end conceptual framework that can serve as a reference architecture for end-to-end deployment. Specifically, we discuss key aspects of big data analytics (storage solutions, platforms like Hadoop and Spark, NoSQL databases), preprocessing approaches (dimensionality reduction, instance selection, noise handling), ML algorithms (supervised, unsupervised, and emerging deep learning paradigms), and deployment considerations (monitoring, continuous integration, and versioning). By gathering and integrating these elements, our survey provides a comprehensive overview of existing solutions, clarifies the design choices developers must consider, and identifies research gaps that remain to be addressed.