Sign language is the fundamental mode of communication for people with hearing and speaking disabilities worldwide. The goal of automating sign language understanding is to establish inclusive visual communication to facilitate conversations between sign language users and the wider community. This paper deals with the problem of understanding ASL alphabet gestures and proposes a two-stage deep learning and computer vision-based framework for ASL alphabet recognition and classification into 26 distinct classes. Existing systems struggle with variations in signer appearance, complex backgrounds and limited high-quality data. The proposed method attempts to address these challenges by experimenting with features and classification techniques and uses a learning-based automatic annotation method for labelling data. This work contributes to the ongoing effort of building automated camera-based systems for inclusive communication that can potentially revolutionize visual communication.