Weed detection in potato crops is crucial for agricultural security as it helps maintain the integrity and productivity of the agricultural system. Controlling weeds is essential to prevent competition for resources, such as nutrients and water, and to protect the desired crop from potential diseases or pests that weeds may harbor. Efficient weed detection and management contribute to the overall security and sustainability of agricultural practices by ensuring optimal crop growth and yield. Inefficient and unsuited for connection with smart, conventional methods of weed management are widely used. The automation systems to identify and classify weeds potentially play a significant role, especially in terms of contributing to increased agricultural yields. Using RGB photos of potato crop fields, the present research investigated the potentiality of deep learning-based approaches (InceptionV3, and Xception) for the purpose of weed identification and classification. The models that were chosen ranged from 94.5% to 97.7% based on the overall accuracy. By achieving 97.7% accuracy, correctly identifying the target variable in 97.7% of cases, 98.5% precision, meaning 98.5% of its positive predictions were correct. Finally, its recall rate was 97.8%, indicating it identified 97.8% of all true positive cases, InceptionV3 demonstrated best performance among the models on a 25-epoch, 35-epoch and 45-epoch with 32-batch sizes.