Creating Recurrent Neural Networks
摘要
Recurrent Neural Networks (RNNs) form the backbone of modern sequence modeling, enabling deep learning models to capture temporal dependencies in data such as text, time series, and speech. In this chapter, we first motivate the need for RNNs and describe common architectures—many-to-many, one-to-many, and many-to-one—highlighting when to use each pattern (e.g., named-entity recognition, image captioning, sentiment analysis). We then cover the essential preprocessing step of tokenization, from character-level encoding to padded, one-hot representations. Next, we derive the math and code for RNN forward propagation, detailing single-cell operations, hidden-state updates, and full-sequence passes. We tackle training challenges—vanishing and exploding gradients—and experiment with remedies such as gradient clipping and better weight initialization. Finally, we explore creative text generation with temperature-controlled sampling and walk through an end-to-end case study: training an RNN to invent novel botanical family names. By the chapter’s end, readers will be able to build, train, and apply RNNs for a wide range of sequence-to-sequence tasks.