Transformers

Transformers extend traditional RNNs and LSTMs by introducing self-attention, the core mechanism behind their success. Unlike earlier models that process words sequentially, transformers assign varying attention weights to words based on their relevance to one another while encoding them into fixed-size vector representations.