<p>Windowing is the process in which the pixel values of an image are remapped to enhance the contrast of the regions of interest. For example, the width and level of the contrast window are adjusted to change the dynamic range and brightness of the CT image accordingly. Although deep learning is widely applied to analyzing medical images, selecting contrast windows for deep learning-based medical image analysis has not been fully explored. In this paper, we propose a data-driven windowing method which learns and suggests multi-contrast windows for the subsequent segmentation models to improve the prediction performance. The proposed method also provides a contrast-enhanced image to visualize which windows are relatively important for the subsequent deep learning model’s prediction task. Through the experimental studies including liver tumor segmentation in CT images and abdominal organ/brain tumor segmentation in MRI images, we demonstrate that our multi-contrast windowing method enables subsequent models to improve the segmentation accuracy along with task-specific contrast-enhanced images.</p>