In numerous academic work, Artificial Intelligence (AI) has demonstrated remarkable capabilities to detect software vulnerabilities from source code, particularly through transformer-based models like CodeBERT and Llama. However, in practice, these models often demonstrate limited generalization capability when applied to new codebases, raising critical questions about what they actually learn during training and whether they truly understand code semantics or merely recognize superficial patterns. By design, their black-box nature further complicates this issue by obscuring their decision-making process. This paper introduces VulFocus, an interactive visualization tool that facilitates interpreting the decision-making process of transformer-based models in vulnerability detection. VulFocus implements two interpretation approaches: attention-based analysis that highlights code regions of focus, and masked language modeling that identifies surprising or unexpected code segments. Our evaluations demonstrate that these visualizations help security analysts and researchers understand the reason behind model decisions, validate findings, assess model generalization capabilities, and understand ineffectiveness of variable-renaming based adversarial attacks. VulFocus bridges the gap between AI capabilities and human interpretability in software vulnerability detection, enabling more effective collaboration between automated systems and security analysts.