Transfer Learning and Transformer Technology

Transfer learning is a commonly used deep learning model to minimize computational resources. This chapter explores: (1) Transfer Learning (TL) against traditional Machine Learning (ML); (2) Recurrent Neural Networks (RNN), a significant component of transfer learning with core technologies such as Long Short-Term Memory (LSTM) Network and Bidirectional Recurrent Neural Networks (BRNNs) in NLP applications, and (3) Transformer technology architecture, Bidirectional Encoder Representation from Transformers (BERT) Model, and related technologies including Transformer-XL and ALBERT technologies. This chapter also serves as the conceptual basis for Workshop#6: Transformers with spaCy and Tensorflow in Chap. 15 .