Investigating the Use of Snowballing on Q&A Websites
摘要
Background: The use of grey literature (GL) has grown in software engineering research, especially in studies that consider questions and answers (Q&A) websites, since software development professionals widely use them. Although snowballing (SB) techniques are standard in systematic literature reviews, little is known about how to apply them to GL reviews. Aims: This paper investigates how to apply SB approaches on Q&A websites to identify new valid discussions for analysis during the exploration of such sites. Method: In previous studies, we compiled and analyzed a set of Stack Exchange Project Management (SEPM) discussions related to software engineering technical debt. Those studies used a data set consisting of 108 valid discussions extracted from SEPM. Based on this start data set, we perform forward and backward SB using two different approaches: link-based and similarity-based SB. We then compare the precision and recall of those two SB approaches against the search-based approach of the original study. Results: In just one SB iteration, the approaches yielded 291 new discussions for analysis, 130 of which were considered valid for our study. This represents an approximate 120% increase in recall compared to the original data set. The SB process also yielded a similar rate of valid discussion retrieval when compared to the search-based approach (precision). Conclusion: This paper provides guidelines on how to apply two SB approaches to find new valid discussions for review. To our knowledge, this is the first study that analyzes the use of SB on Q&A websites. By applying SB, it is possible to identify new discussions, significantly increasing the relevant data set for a GL review.