Managing Large Twitter Datasets

This chapter provides a thorough discussion of Twitter/X corpora in terms of compilation and management. Twitter corpora differ from other types of corpora in many aspects, as they are composed of a very large number of very small documents (tweets), each with a slew of metadata, that can be downloaded through scripts that make use of available APIs, which calls for certain tools and techniques. The type of language used in social media is also very different from other, more standard genres, both in form and content. When this is coupled with large-size corpora, effective sampling techniques are necessary, which are discussed at length in this chapter. Finally, a description is given of using geotagged data and subcorpora creation and management.