The physical condition of driver has a significant impact on driving safety, and its monitoring has been widely studied. The progressive development of the Internet of Things (IoT) has motivated the health care system to employ the sensor to monitor the driver body. The collected physiological data can also be used by the On Board Unit (OBU) of vehicle to assess the driver’s physical condition and ensure driving safety. In the case of Internet of Vehicles (IoV) integration with Body Sensor Networks (BSN), the communication security issues must be addressed. Accordingly, an authentication protocol is needed to enable the vehicle’s OBU to securely obtain the physiological data collected from the body sensors of driver. In this paper, a cross-domain authentication protocol for secure integrating IoV and BSN is proposed. The trusted authority participates in the authentication process to reduce the computational cost of body sensor and OBU. In addition, the proposed protocol uses pseudo identity to ensure entity anonymity.