Depalletizing, the process of unloading objects from pallets, is a critical task in warehouse automation. Robotic systems face challenges due to the diverse shapes and sizes of objects, often leading to failed grasps in cluttered environments. Pushing actions offer a promising solution to facilitate object singulation. In this work, we employ reinforcement learning (RL) for obtaining optimal pushing policies in depalletizing. We formulate the singulation task as goal reaching, where the target object is moved towards a goal area free from obstacles. Our approach, Goal-FCN (Goal-Driven Fully Convolutional Network), combines feature extraction with a fully convolutional neural network to predict effective pushing actions based on RGB-D images and scene segmentation. Experiments demonstrate the efficacy of our RL-based approach in achieving high success rates in challenging depalletization scenarios.