<p>Sentiment analysis is known to be the useful technique that is applicable for contextual mining, this makes it simple to extract textual information based on context. Although various features extraction techniques like word2vec, glove, fastext is used to analyse and extract the features and context from the text but they need large dataset for training. They also have a fixed vocabulary set for which it generates vectors due to which lot of information is lost. This paper aims to improve the performance of the BERT (Bidirectional Encoder Representation of Transformer) algorithm on two different datasets using a novel feature selection technique called Dynamic Attention-Based Feature Selection (DAFS) in combination with a Genetic Algorithm (GA) to optimize the attention weights for feature selection and hyperparameters, such as the number of layers in the Advance BERT model or the learning rate. The main idea behind DAFS is to adaptively select the most informative features based on the task, rather than relying on a fixed set of features. DAFS achieves this by learning attention weights that determine the importance of each feature for the task. The proposed technique achieved promising results, the developed approaches are expected to be applied to address various challenges in social media and commercial contexts.</p>