In the era of rampant digital manipulation, the need for robust forgery detection techniques for verifying the authenticity of the digital contents is imperative. Even though forgery detection in the images has received significant attention, the success in case of video de-forging (i.e., video manipulation detection techniques) has been less explored. This is because there exists a certain ghost phenomenon in the manipulated video that occurs when videos are altered and this makes the anti-forensic measures unsuitable for verifying their authenticity. Ghost phenomena in video manipulation refers to the artifacts created by multiple compression and decompression processes, resulting in faint, residual images, or trails of previous frames appearing in subsequent frames. These artifacts manifest as translucent duplicates or blurs, degrading the quality and fidelity of the video. Despite the lack of appropriate video forensic techniques, the paper highlights and analyzes the prevalence of ghost phenomenon associated with single video de-forging. Our objective is to provide insights into the ghost phenomenon associated with single video de-forging and thereafter propose a novel approach for forgery classification leveraging Deep Convolutional Neural Networks (DCNN). Our proposed method utilizes automatically learn discriminative features from manipulated images, enabling accurate classification of authentic and forged content. Evaluation on our own created dataset and available REWIND dataset showcases the superior performance of our approach, achieving an impressive accuracy of 96.73% and 95.31%, respectively. This indicates the effectiveness of our method in discerning subtle manipulations, making it a valuable contribution to the field of multimedia forensics.