Higher education institutions generate large amounts of data from various sources, including student information systems, learning management systems, administrative records, etc. For this reason, systematic data management and use are essential in decision-making and strategic planning. However, data management practices are currently struggling to handle the growing volume and complexity of data, making it challenging to obtain useful information for quick decision-making. Furthermore, integrating heterogeneous data sources and processing data in real-time while maintaining data quality remains challenging. Therefore, in the context of Malaysian education, this study proposes a large-scale big-data architecture designed specifically for the Malaysian Ministry of Higher Education (MoHE). It uses Apache technologies such as Apache Hadoop, Apache Spark, Apache Kafka, Apache NiFi, and Apache Cassandra to provide comprehensive solutions for data integration, real-time processing, and data quality management. This architecture aims to improve the efficiency and effectiveness of higher education data management in making better decisions. This innovative approach enable MoHE to can make more accurate decisions based on multiple data sources, enable real-time analysis, and ensure robust data governance, leading to a more responsive and data-driven educational environment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Scalable Big Data Architecture: Improving Data Management at MoHE

  • Zhamri Che Ani,
  • Fauziah Baharom,
  • Haslina Mohd,
  • Yuhanis Yusof,
  • Mohamed Ali Saip

摘要

Higher education institutions generate large amounts of data from various sources, including student information systems, learning management systems, administrative records, etc. For this reason, systematic data management and use are essential in decision-making and strategic planning. However, data management practices are currently struggling to handle the growing volume and complexity of data, making it challenging to obtain useful information for quick decision-making. Furthermore, integrating heterogeneous data sources and processing data in real-time while maintaining data quality remains challenging. Therefore, in the context of Malaysian education, this study proposes a large-scale big-data architecture designed specifically for the Malaysian Ministry of Higher Education (MoHE). It uses Apache technologies such as Apache Hadoop, Apache Spark, Apache Kafka, Apache NiFi, and Apache Cassandra to provide comprehensive solutions for data integration, real-time processing, and data quality management. This architecture aims to improve the efficiency and effectiveness of higher education data management in making better decisions. This innovative approach enable MoHE to can make more accurate decisions based on multiple data sources, enable real-time analysis, and ensure robust data governance, leading to a more responsive and data-driven educational environment.