Crafting Long Short-Term Memory Networks
摘要
Recurrent neural networks (RNNs) often struggle to learn long-term dependencies due to vanishing gradients. This chapter introduces long short-term memory (LSTM) networks, a powerful RNN variant designed to overcome those limitations by augmenting the traditional hidden state with a dedicated cell state and three gating mechanisms—forget, update, and output gates. We begin by detailing the internal architecture of a single LSTM cell, derive its forward-propagation equations, and learn how sequences of these cells capture information over extended time horizons. Next, we explore the encoder–decoder paradigm, illustrating how LSTMs power modern sequence-to-sequence tasks like machine translation, where an encoder compresses an input into a context vector and a decoder generates the target sequence from it. We then walk through training procedures—loss computation, backpropagation through time, and parameter updates—highlighting practical considerations for building and tuning deep LSTM models. Finally, we craft an end-to-end LSTM system for numerical translation from French to English words, showcasing data preparation (tokenization, padding, one-hot encoding), model training, and decoding. By the chapter’s end, readers will understand both the theory and practice of crafting LSTM architectures for long-sequence modeling tasks.