Multimodal Analysis of Hot Lap Recordings for the Determination of Vehicle Speed and Engine RPM Profiles Using Audio and Video Processing
摘要
In the context of competition-oriented vehicle development, the analysis of publicly available Hot Lap videos presents a promising approach for obtaining technical insights into competitor vehicles without direct access to internal sensor data. This paper introduces a multimodal methodology for the non-invasive reconstruction of vehicle speed and engine speed through combined video and audio processing. Optical character recognition is employed to extract speed information from on-screen video overlays, while engine speed is derived from spectral analysis of the audio track using Short-Time Fourier Transform and subsequent tracking of the fundamental frequency. The methodology was validated using two vehicles—the Audi RS3 Limousine and the Renault Mégane R. S. Trophy-R. A correlation of 77% was achieved between extracted and measured engine speeds for the Audi and 62% for the Renault, each within a ±1000 rpm tolerance band. Accuracy was primarily influenced by microphone placement, video overlay quality, and assumptions regarding gear selection. Despite these limitations, the method demonstrates significant potential for technical competitor benchmarking. Future enhancements, such as adaptive gear estimation models and more robust signal processing techniques, may further improve the precision of performance reconstruction based on publicly available video sources.