How To Test The Machine Learning Algorithms That Are Common in Big Data Applications

The extensive use of information and, thereby, also the application of big data (BD) technologies, are some of the biggest influencing factors in today’s society. However, due to the sheer deluge of data, it is not feasible to turn them into usable information in a manual fashion. Instead, automated approaches are required, which makes machine learning (ML) algorithms an important part of the corresponding technical ecosystem. Yet, besides the pure provisioning of the algorithms, it is also necessary to make sure the delivered quality is sufficient. Hence, the testing of the ML algorithms in the BD context with its specific challenges is highly important. For this reason, in the publication at hand, based on previously identified BD standard use cases, the common ML applications are identified and it is discussed, how they can be tested, providing future researchers and practitioners in the domain with valuable insights on how to create better quality BD applications.