SmartLawAnnotator: A Machine Learning-Based Annotation Recommendation for Judgment Prediction

In the past few years, active learning-based modeling has brought notable changes in various industries and operations. In the legal domain, the adoption of these recommendations and prediction-based machine learning (ML) or deep learning (DL) systems has proven helpful. However, not much research has been carried out on the legal annotation recommendations or has been restricted to a few models only. Also, research in the Indian legal domain has been limited. This study applies ML to the problem of legal annotation recommendations for the Indian legal domain for criminal bail applications. Extensive research experiments were conducted with 17 popular machine learning models, and results were compared using standard techniques. To implement this open-source active learning-based legal annotation recommendation system, ‘SmartLawAnnotator’ was developed. Around 2000 samples were labeled with help from legal experts using this system. Based on the results obtained, it is established that support vector machine models outperform most of the models with an accuracy of 82% when the sample size was less. When the sample size started to grow, ensemble-based models like the random forest model performed better with 86% accuracy. Both the source code of the system and the dataset developed were made publicly available.