Ghost Data in Data Quality Management

Drawing an analogy to the concept of ghost particles in physics, the management of unobserved or undetected data—effectively “gauging” the unknown—constitutes a persistent challenge for data scientists. This chapter introduces the concept of ghost data within the domains of statistics, computer science, and economics. Furthermore, we will delve into the intricacies of procuring high-quality data, which encompasses a multitude of stages, including data collection, annotation, purification, de-identification, aggregation, and analysis.