A Passenger Detection and Action Recognition System for Public Transport Vehicles

In the past few years, advances of autonomous driving technologies have made significant progress. One successful deployment with great potential is for the self-driving public transportation. Consequently, the safety of passengers under unmanned operations is an important issue for the future development. In this paper, we propose a vision system for the detection, counting and action recognition of passengers on board a bus. To have a complete coverage of the passenger area, a pair of cameras mounted on the ceiling is adopted for image acquisition. A new cross-camera CNN structure is proposed for passenger pose recognition and action classification. To deal with the double counting problem caused by the overlap of multiple viewpoints, a region prediction algorithm is developed for matching validation. The experiments carried out on a minibus have demonstrated the feasibility of our proposed techniques.