Development and Deployment Challenges of Machine Learning Systems

In this research, a total of 30 small- and medium-sized enterprises (SMEs) and large companies based in Finland and Ireland were surveyed on the perceived development and deployment challenges of machine learning (ML) systems. Additionally, comparisons were made between SMEs and large companies to assess if any important differences exist. Previous research efforts have focused largely on identifying challenges and possible related solutions within the context of large companies. This research contributes to this stream of literature by aiming to identify the specific development and deployment challenges within the context of SMEs, and also to provide further research directions within this context. As significant findings for SMEs (n = 23, p<0,05), it was observed that risk- and resource estimation, data annotation, selection of an optimal modeling technique, determination and optimization of the required performance for ML models, validation of model results, monitoring for data- and concept drift, as well as establishing and maintaining governance procedures are perceived as challenging areas. As additional findings, further challenges and sources of risk were synthesized from the free text answers provided by the survey respondents. Many governments are specifically addressing the importance of SMEs in their artificial intelligence strategies. On top of providing new research directions, these findings also thus allow for more targeted regional and local development efforts.