On the uniqueness of AntiVirus labels: How many labels do we need to fingerprint an AV?

The biggest drawback of AntiViruses (AVs) experiments is label heterogeneity–each AV labels the same samples very distinctly. Whereas AV labeling issues have been well studied from the sample point of view, they have not been widely studied from the AV perspective, i.e., to what extent label diversity allows AV identification. Thus, we question: (1) How unique among all the AVs are the labels produced by the same given AV? (2) Can we fingerprint AVs based on their assigned labels? and (3) How many labels are required to fingerprint an AV? In this work, we answer these questions via experiments with a dataset of 720000 AV-assigned labels for Windows malware spread over 15 years (2006-2020). We discovered that: (1) AVs can be fingerprinted by their assigned labels with 100% accuracy in many cases; (2) AVs can be fingerprinted with a confidence score of 99% using only 1% of the dataset; (3) AV fingerprinting rates vary over time, as the label changes caused by the AV updates have a key effect on AV recognition, causing some AV models to lose their ability to recognize their AV generated labels over time; and (4) Android AVs can be fingerprinted the same way as Windows AVs, but that Linux labels are harder to be grouped. We expect our work might shed light on the label heterogeneity problem, incentivize further developments to mitigate it, and provide future works with data to support their design decisions.