Phone use can distract drivers from driving safely. This research presents an approach to detecting driver distraction caused by handheld mobile phones, employing a deep learning model that combines Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM). The study collected physiological signals and head motion data in a simulated driving environment using a motion platform, capturing data such as heart rate, breathing rate, galvanic skin response, and skin temperature, alongside accelerometer and gyroscope metrics for head movements. This study is novel as it employs motion platforms to provide realistic driving motion sensation aiming to explore how phone distractions affect physiological reactions and head movement data, specifically when integrating advanced deep learning methods for distraction detection purposes. Notably, the CNN-LSTM model was rigorously optimized using Bayesian Optimization for hyperparameter tuning, ensuring the highest efficacy in processing the multi-modal data. The performance of the CNN-LSTM model was evaluated against several classical machine learning models and a deep feedforward neural network using 5-fold cross-validation, demonstrating superior performance metrics. These metrics are accuracy of 99.41%, precision of 99.19%, recall of 99.60%, and an F1 score of 99.39%. These results highlight the model’s robustness and its potential integration into driver-assistance systems to enhance road safety by providing timely alerts to distracted drivers.