Publicly available datasets are crucial for evaluating coronary artery segmentation models. However, annotation inconsistencies can affect performance assessments and lead to the omission of clinically relevant insights. In this study, we benchmark multiple topology-aware segmentation methods against each other and compare them with a non-topology-aware approach using ASOCA, the most widely used dataset. To ensure a clinically meaningful evaluation, we assess both traditional and topology-based segmentation performance, incorporating expert medical feedback to stratify the coronary artery tree into anatomically relevant segments. Our findings show that topology-aware segmentation methods perform similarly on primary segments, the most clinically significant ones, suggesting that topology-aware losses provide limited benefits in these cases. Performance differences become more pronounced in secondary and tertiary segments, where annotation inconsistencies and dataset biases have a greater impact on segmentation accuracy than the choice of method. These results highlight the need for clinically-informed benchmarking strategies. Future research should focus on developing high-quality datasets and perform segment-specific performance analyses. Aligning evaluation methods with clinical decision-making is key to bridging the gap between model assessment and real-world applicability.