Extraction of valuable insights from online customer reviews helps in better decision-making process. While sentiment analysis serves this purpose, it faces certain problems like review bias where sentiments are inclined excessively positive or negative leading to reduced accuracy. In this study, we propose a system that utilizes word embeddings in order to capture semantic relationships and integrate a CNN-BiLSTM model to detect sentiment of the review more effectively. Topic-based attention mechanism is incorporated in the proposed system to get more information about the predefined context culminating more precise analysis. Performance analysis of the proposed system is performed using two datasets namely Amazon Reviews dataset and Yelp dataset employing a number of performance metrics. Experimentation results indicate the effectiveness of the proposed system to reduce the effect of review bias and enhance the overall accuracy. This research is advancement in the field of sentiment analysis making use of advanced Deep Learning (DL) techniques, enhancing the understanding of customer sentiment.