How Challenging is a Challenge for SLAM? An Answer from Quantitative Visual Evaluation

SLAM (Simultaneously Localization and Mapping) is the fundamental technology for the application of unmanned intelligent systems, such as underwater exploration with fish robots. But various visual challenges often occur in practical environments, severely threaten the system robustness. Currently, few research explicitly focus on visual challenges for SLAM and analyze them quantitatively, resulting in works with less comprehensiveness and generalization. Many are basically not intelligent enough in the changing real world and sometimes even infeasible for practical deployment due to the lack of accurate visual cognition in the ambient environment, as many animals do. Inspired by visual perception pathways in brains, we try to solve the problem from the view of visual cognition and propose a fully computational reliable evaluation method for general challenges to push the frontier of visual SLAM. It systematically decomposes various challenges into three relevant aspects and evaluates the perception quality with corresponding scores. Extensive experiments on different datasets demonstrate the feasibility and effectiveness of our method by a strong correlation with SLAM performance. Moreover, we automatically obtain detailed insights about challenges from quantitative evaluation, which is also important for targeted solutions. To our best knowledge, no similar works exist at present.