Data Collection and Analysis: The Foundation of Evidence-Based Research in Various Disciplines

Data has become one of the most valuable assets for organisations in all industries. Whether you work in IT, banking, e-commerce, retail, education, hotel management, or any other industry, data is critical to knowing how the organisation is operating, where improvements can be made, and how they can scale their operation. Data collection is the systematic process of acquiring and measuring information on variables of interest in order to answer specified research questions, test hypotheses, and evaluate outcomes. The fundamental reason for maintaining data integrity is to aid in the detection of flaws in the data gathering process, whether intentional (planned falsifications) or unintentional (systematic or random errors). This chapter gives a detailed summary of data collection, scrappers, crawlers, and APIs. It also gives a view of data pre-processing, data annotation, data evaluation, and also highlight its various challenges and issues.