Deep neural networks in IoV must meet stringent requirements for real-time performance, low latency, lightweight design, and low power consumption. However, most existing deep neural networks are challenged by their complex structures, which demand significant time and computational resources for feature extraction. This limitation hinders their application on portable devices, particularly in the context of in-vehicle image and video processing. To address this, we introduce GL-CrackNet, a lightweight deep neural network tailored for crack detection on portable devices. GL-CrackNet is based on DeepLabv3+ and includes an encoder and decoder. The encoder integrates the GhostNet embedded network and two CCAM modules to capture a comprehensive representation of features. The decoder employs RFB and Dense Aggregation modules for focused feature processing and fusion. On a custom dataset, GL-CrackNet achieves a mIoU of 80.93%, surpassing the baseline by 0.9%–1.5%. This demonstrates a significant improvement in crack detection efficiency while maintaining low storage and power consumption.