Towards Generic Malware Unpacking: A Comprehensive Study on the Unpacking Behavior of Malicious Run-Time Packers

The presence of packing techniques in malicious software remains a significant obstacle in malware analysis. Consequently, numerous research efforts have emerged with the objective of developing a generic methodology to unpack malware. However, these unpacking methodologies often rely on assumptions about the capabilities of packers. These assumptions include factors such as the origin of memory sources, code-writing techniques used to fulfill packing capabilities, the number of packing layers used, the persistence of code within memory, and the clear distinction between packer and malware code. In our paper, we aim to advance the state-of-the-art by addressing these underlying assumptions associated with malware unpacking. Based on these assumptions, we formulate five research questions to be addressed in a study on the packer capabilities found in a real-world Windows malware and clinical data set consisting of off-the-shelf packers. The answers deduced from our study demonstrate that the majority of common generic unpacking methodologies in the literature show significant blind spots, with the notable exception of the Renovo methodology and its derivatives.