An efficient single-stage ISP for smartphones using global context residual dense and residual channel attention modules
摘要
The utilization of smartphone cameras to capture photographs is immensely popular in the world. Smartphone image signal processors are used to produce high-quality images. The field of Image Signal Processing (ISP) in smartphone cameras involves the application of various techniques and algorithms that process the raw image data acquired by the smartphone camera sensor into a high-quality Red, Green, and Blue (RGB) image. The visual difference between images captured by smartphone cameras and Digital Single Lens Reflex (DSLR) cameras can be attributed to the constrained sizes of smartphone camera sensors and lenses. To address the existing disparity in visual differences of these devices, there is a need to redesign the smartphone ISP with the aim of reconstructing the good quality of the captured images in real time. This work proposes a single-stage end-to-end deep-learning model that can replace most complex ISP pipelines of smartphone cameras. The training of the proposed model is independent of the sensor and optics employed in a specific device. The proposed single-stage ISP pipeline for smartphone cameras uses the Global Context Residual Dense (GCRD) module, the Multiple Convolution Block (MCB) module, and the Residual Channel Attention (RCA) module. The GCRD module is used to learn the residual information which helps in color mapping of the raw image to the corresponding RGB image. At the same time, the MCB module with multiple convolution blocks consisting of layers of different kernel sizes focuses on the fine-grained details of the image. Further, the RCA module used in the proposed work consists of a very high deep trainable network that adaptively learns more beneficial channel-wise features simultaneously. By combining these modules, the pipeline achieves a synergistic effect that helps in balancing global context, local refinement, and feature prioritization, enabling superior performance in complex ISP operations. This work evaluates the proposed model on two datasets, i.e., the Zurich RAW-RGB (ZRR) dataset consisting of the Huawei P20 smartphone captured raw images and the Well-Aligned dataset, which consists of two smartphones, iPhone 6 S (high-end) and Mi-Z6 (low-end) camera captured images. The proposed model is a lightweight model in terms of the number of hyperparameters, which makes it suitable for real-time processing in smartphones. The experimental results show that the proposed model outperformed the benchmark models in terms of output image quality and computational time. This makes the proposed model highly suitable for real-time smartphone ISP. Experimental evaluation ensures the proposed model’s robustness, scalability, adaptability, and resource efficiency in various scenarios. Additionally, this work also evaluates the qualitative and quantitative results along with the ablation study of the modules.