State-of-the-art machine learning models have continuously improved over recent years, leading to increasingly high-performing models. Simultaneously, it has become increasingly difficult to infer insights in model behavior, as models continuously increase in complexity. This paper aims to increase the explainability of the HAABSA++ model, a state-of-the-art machine learning algorithm that combines a domain sentiment ontology with a deep learning model using contextualized word embeddings, designed for aspect-based sentiment analysis. The model is trained and evaluated on the SemEval 2016 restaurant reviews dataset. For model explainability we propose two SHAP approaches. The first approach, SHAP model 1, applies SHAP after BERT word embeddings are generated, measuring the contribution of each embedded word to the sentiment prediction. The latter approach, SHAP model 2, applies SHAP before BERT word embeddings are generated. SHAP model 2 appears to be better at finding words that individually contribute most towards a sentiment prediction since it generates new word embeddings for each subset of the original sentence.