An initial step in the data mining pipeline is data pre-processing. The chapter starts with an introduction of key concepts such as data dimensionality, data types, and levels of measurement, providing examples and implications for data analysis. Then, data errors and inconsistencies are explored, including their sources—measurement errors, changeable collection methods, and data entry mistakes. The chapter outlines data cleaning techniques, such as removing duplicates, handling missing data, and cleaning text data. It also discusses data transformation methods and strategies for addressing challenges such as outliers and group imbalances. The Python-based lab exercise guides readers in using the Anaconda AI assistant to clean data from a real-world basketball game visitors’ survey.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Data Pre-processing

  • Andrei P. Kirilenko

摘要

An initial step in the data mining pipeline is data pre-processing. The chapter starts with an introduction of key concepts such as data dimensionality, data types, and levels of measurement, providing examples and implications for data analysis. Then, data errors and inconsistencies are explored, including their sources—measurement errors, changeable collection methods, and data entry mistakes. The chapter outlines data cleaning techniques, such as removing duplicates, handling missing data, and cleaning text data. It also discusses data transformation methods and strategies for addressing challenges such as outliers and group imbalances. The Python-based lab exercise guides readers in using the Anaconda AI assistant to clean data from a real-world basketball game visitors’ survey.