Weeds poses a significant challenge in agriculture, competing with crops for resources and reducing yields. Traditional weed management methods, such as herbicide application or manual removal, are often labor-intensive, time-consuming, and may have adverse environmental impacts. In recent years, robotic systems equipped with computer vision have emerged as a promising solution for automating weed detection and removal tasks. This study proposes a novel approach combining a robotic arm with the Mask R-CNN (Region-based Convolutional Neural Network) algorithm for efficient weed detection and targeted removal in agricultural fields. Experimental results demonstrate the effectiveness and efficiency of the proposed system in weed detection and removal tasks. The integration of Mask R-CNN enables precise localization of weeds, reducing false positives and minimizing the risk of inadvertent damage to desirable plants. Based on weed parameters the model predicts the weed plants with accuracies ranging from 80 to 98% telling it is a weed with the region mapped with colors and blocks. Furthermore, the robotic arm facilitates targeted weed removal, thereby reducing the reliance on herbicides and manual labor. Overall, this research contributes to the advancement of automated weed management systems, offering a sustainable and environmentally friendly approach to weed control in agriculture. The combination of robotic technology and deep learning-based image analysis holds great promise for improving crop productivity and reducing the ecological footprint of farming practices.