Analyzing Classifications with Low Trust in Ground Truth Premises: The Case of Bots and Humans in Advertising-Related Traffic
摘要
We first describe the selection of methodological approaches, applied to the problem of distinguishing the human and bot-generated traffic, related to web advertising, in a commercially motivated project. Then, it is explained how a set of different behavior patterns was identified, concerning the distinguished web users, humans or bots. In this context, the issue is discussed of the possibility of correct labelling of these users with the help of an ad hoc expert-designed instrument (the labelling discerning only humans from bots). It turns out that while there are behavior patterns assigned in an indisputable manner to either humans or bots, and the expert-designed instrument distinguished them clearly, there are other ones, hardly distinguishable, casting doubt on the actual correctness of the instrument in this respect. The results from a number of analyses are shown for the critical subsets and their characteristics. Some conclusions of much wider applicability are drawn.