This paper presents an adaptive multi-stage sensor fusion (AMSF) approach under the neuro-symbolic framework for a multi-modal ranging system. In particular, the ranging system integrates LiDAR with a stereo camera for robust distance measurements against various adverse environmental conditions. A multi-stage approach that consists of 1) raw data processing, 2) data integrity verification, 3) individual performance evaluation, and 4) decision fusion is proposed to enhance the diversity, flexibility, and performance of the AMSF system. Furthermore, an adaptation mechanism is introduced to adjust the fusion strategy based on the environmental conditions. Moreover, the neuro-symbolic (NS) framework integrates the advantages of neural networks (NNs) and symbolic reasoning to improve the explainability and reliability of the adaptation mechanism by incorporating human expert knowledge into the decision-level fusion. To account for the uncertainty in the NS inference, variational dropout is incorporated into the NNs for uncertainty quantification and evaluation as well as robust decision-making. Experiments on the dataset collected from our ranging system demonstrate the proposed approach can improve the ranging precision achieved from single-modal sensors and enhance the fusion performance as compared with pure deep learning (DL)-based sensor fusion algorithms.