Nearly 80% of global data exists as unstructured textual data. This chapter focuses on the essentials of transforming the texts into structured numerical data suitable for the identification of recurrent patterns, the process known as text mining. Unlike numerical data, text data is inherently noisy, messy, and unstructured. It is laden with typos, jargon, dialects, nuances, ambiguities, and subtleties that can easily be lost or misinterpreted in the computer-based text interpretation process. Furthermore, text data is high-dimensional, which complicates the analysis. The chapter discusses common text pre-processing methods, text visualization, and conversion of texts to numbers, such as word embedding. Text similarity measures and document clustering are discussed to provide examples of further analytic approaches. The Python lab includes a comparative analysis of U.S. and Swiss hotel guest reviews, showcasing text preprocessing, visualization, and clustering. The chapter concludes by emphasizing the integration of statistical and computational methods to analyze text, laying the groundwork for advanced text mining algorithms in subsequent chapters.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text Mining Essentials

  • Andrei P. Kirilenko

摘要

Nearly 80% of global data exists as unstructured textual data. This chapter focuses on the essentials of transforming the texts into structured numerical data suitable for the identification of recurrent patterns, the process known as text mining. Unlike numerical data, text data is inherently noisy, messy, and unstructured. It is laden with typos, jargon, dialects, nuances, ambiguities, and subtleties that can easily be lost or misinterpreted in the computer-based text interpretation process. Furthermore, text data is high-dimensional, which complicates the analysis. The chapter discusses common text pre-processing methods, text visualization, and conversion of texts to numbers, such as word embedding. Text similarity measures and document clustering are discussed to provide examples of further analytic approaches. The Python lab includes a comparative analysis of U.S. and Swiss hotel guest reviews, showcasing text preprocessing, visualization, and clustering. The chapter concludes by emphasizing the integration of statistical and computational methods to analyze text, laying the groundwork for advanced text mining algorithms in subsequent chapters.