To ensure the safety and stability of mobile robots operating alongside humans in industrial environments, the robot is typically equipped with vision and depth sensors to perceive the surroundings and required to maintain appropriate speed and safe distances by identifying obstacles ahead. However, the traditional vision-based obstacle detection methods suffer the drawbacks of unstable obstacle detection in complex environments and imprecise distance estimation of the obstacles. To address these issues, this paper proposes an innovative real-time obstacle detection and safe operation framework using the complementary RGB and depth measurements for industrial autonomous mobile robots. To enhance the effectiveness of obstacle detection and relative depth estimation, RGB and depth measurements are integrated in an innovative manner. The spatio-temporal hybrid deep learning network is built to segment obstacles and accurately determine their depth using the Euclidean clustering. An adaptive control strategy is designed to dynamically adjust the robot’s speed, maintaining a minimal safety distance to ensure the safety of both the robot and the surrounding obstacles. Meanwhile, we implement a temporal information aggregation tracking model to detect falls among workers. Upon detecting a fall, the robot will immediately halt its operations. If the fall duration exceeds a predefined threshold, an alarm will be triggered to ensure worker’s safety. The algorithm has been tested on a specially developed mobile robot platform. The experimental results highlight the competitive performance of our method.