Monocular depth estimation is a commonly applied part of UAV obstacle avoidance. At the same time, the monocular depth estimation evaluation methods lag behind. The traditional monocular depth estimation methods do not consider the physical properties of obstacle avoidance. At the same time, the evaluation of complete obstacle avoidance systems is complicated and does not directly focus on the monocular depth estimation model. In this study, we propose an in-between evaluation method. Although our method focuses on the monocular depth estimators used, it still considers the physical properties of obstacle avoidance. We also provide practical guidance on how to use our evaluation method in real life. The guideline also describes the connection to other related topics, such as multi-objective optimization. The research concludes with a case study. Here, we investigate the effect of object insertion on the predictions of a monocular depth estimator, focusing on obstacle avoidance. The code and our results are published at: https://github.com/mntusr/mde_oa_eval .