High-quality data annotation is critical for the success of machine learning models, particularly in supervised learning and large-scale projects involving multiple annotators. This paper introduces LS-Dashboard, a comprehensive analytical tool designed to evaluate and monitor annotation projects. LS-Dashboard integrates seamlessly with Label Studio, an open-source annotation platform, to provide real-time insights into annotator performance, task distribution, and agreement metrics. The tool consists of a Python package for backend data analysis and a Streamlit-based app for interactive visualizations. By utilizing two inverted index structures to efficiently retrieve annotation data, LS-Dashboard enables in-depth insights at both the annotation task and annotator levels. It supports classification tasks, offering customizable visualizations for detailed project monitoring.