Multi-level Matching of Natural Language-Based Vehicle Retrieval

Utilizing natural language to retrieve vehicles of specific types and motion states in videos holds great significance for analyzing traffic conditions. But natural language and vehicle video contain rich semantics, including static and dynamic information about vehicles. Additionally, the flexibility of natural language allows for multiple expressions of sentences with identical semantics. To make full use of the information in it, we divide the natural language and video data into different levels and divide them into the representation of overall and local information. We propose information enhancement methods for different data levels, followed by generating embedded representations for layered data using representation learning networks. Finally, the overall cross-modal similarity is calculated by applying weighted measures. Experimental results demonstrate the method’s capability to enhance the accuracy of retrieving vehicles in specific states from videos using natural language.