AutoLabel: Automated Textual Data Annotation Method Based on Active Learning and Large Language Model

In response to the substantial demand for labeled training data in machine learning model training, this study proposes an automatic text data annotation method, AutoLabel, which leverages a large language model and active learning techniques. AutoLabel systematically assists annotators in efficiently accomplishing text labeling tasks. The approach begins by selecting the most representative seed data using the density clustering algorithm and a systematic sampling method. Subsequently, it constructs a prompt dialogue based on the Chain-of-Thought Prompting engineering to stimulate learning in the large language model. Finally, human feedback is introduced to rectify the labeled results. AutoLabel successfully attains a substantial quantity of high-quality labeled samples at the expense of a minimal number of labeled instances. This, in turn, furnishes essential data support for downstream tasks such as entity recognition. Our method demonstrates exceptional performance, achieving accuracy rates of 96.43% and 88.78% on the CoNLL-2004 and ADE datasets, respectively. When compared to conventional supervised deep learning models, AutoLabel exhibits superior accuracy, showcasing improvements of 3.09% and 11.65% over baseline results for the respective datasets.