The visual sign recognition system is to convert sign symbols into speech patterns faces several challenges, due to the variability in sign gestures and body movements across different languages and cultures. These gestures, encoded as visual content, carry specific emotional and contextual motivations, which need to be accurately identified to enable seamless speech synthesis. A significant challenge lies in achieving high accuracy in recognizing sign symbols, especially when presented in diverse image formats or with subtle variations in gestures. Our research presents novel strategies for creating a framework that can manage Sign Language identification, translation, and recognition in the Odia regional language with assigned meanings for particular signs. The model is to recognize sign language by identifying hand gestures in full-body photos. Data augmentation techniques were used to increase the size of the dataset from 300 to 3,000 photos because of its constrained size. A hybrid strategy integrating Convolutional Neural Networks (CNN) and YOLOv8 was used to efficiently recognize and categorize gestures with encouraging outcomes. Promising results were obtained by using a hybrid approach that combined YOLOv8 and CNNs to effectively recognize and classify movements. With an accuracy of 94.5%, the suggested model outperformed standalone YOLOv8 (91.2%) and CNN (82.5%). With a relative improvement of 14.5% over CNN and 3.3% over YOLOv8, this shows how well the hybrid technique captures subtle differences in gestures while preserving real-time speed.