Transformer Models in Natural Language Processing

The development of transformer-based language models brings a paradigm shift in the world of smart applications. The ChatGPT model opened new horizons in the field of natural language understanding and generation. This paper presents a survey on the history of transformer models, on the basic architecture and application areas. The last section is devoted to two use cases experiments on the application of ChatGPT. The first domain relates to Human-Level Programming and the second focuses on the semantic functional parsing of text sentences. The performed analysis demonstrates the big potential in the transformer language models.