In biomedical research, text classification plays a pivotal role when it comes to retrieving important data from large scientific abstract databases. Numerous healthcare fields stand to gain significant advantages from advancements in natural language processing (NLP), particularly within artificial neural networks. Sequential text categorization is highly influenced by Long Short-term memory networks (LSTMs) and recurrent neural networks (RNNs). However, issues such as exploding or vanishing gradients arise during training, impeding their ability to adequately capture long-range associations. Furthermore, these models might not be able to handle the complex semantic relationships included in text data. Recognizing these drawbacks highlights the need for creative solutions that push the limits of natural language processing (NLP) methods in order to more effectively handle the intricacies of biomedical text classification. Recognizing the limitations, efforts are focused on innovative ways to improve text classification in biomedical research, such as transformer topologies and attention processes. The purpose of this work is to investigate how well different LSTM architectures combined with BERT models might improve the understanding and categorization of biologically complicated sequential material in biomedical abstracts. The study emphasizes the dedication to improving techniques and utilizing the advantages of various models to get better results in the complex field of biomedical text classification. Our model employed RNN with LSTM architectures and two variations, comparing their performance with BERT, a state-of-the-art language model acclaimed for its exceptional contextual understanding. This approach explores the effectiveness of different architectures in sequential text classification for biomedical abstracts. The models underwent thorough training on three substantial datasets, yielding promising outcomes. The comparative analysis revealed that our model excels, attaining state-of-the-art accuracy, precision, recall, and F1-score metrics in biomedical text classification. Notably, our model demonstrated robust generalization capabilities, indicating its potential for real-world applications. Utilizing advanced contextual comprehension and augmented hidden units, our adaptive BERT model outperformed competitors, highlighting its efficacy in extracting valuable insights from biomedical abstracts. This firmly establishes its status as a top-performing solution in biomedical research text classification. The models have been extensively trained on three large datasets to achieve promising results.