Learning from Text

We begin with the more technical section of this book by focusing on methods that learn from texts. This subfield of machine learning has seen substantial advancements in the last decade and remains one of the fastest progressing areas. We begin by discussing non-contextual representation learning – a branch of efficient methods enabling learning from texts representing the first wave of substantial improvements across many tasks, from text classification to entity matching and similarity search. We continue by discussing contextual representation learning, the area of machine learning that revolves around large language models, and how they can be used to obtain high-quality, context-dependent representations. This branch of methods remains state-of-the-art for many real-life tasks and is commonly considered as the gold standard when designing machine learning systems.