Large Language models (LLMs) based on the Transformer architecture are designed to understand and generate human-like text by learning patterns and relationships from vast amounts of textual data. These models have been reported to have “sparks” of artificial general intelligence owing to their very attractive properties such as high generalization and performance in unseen data. These models have demonstrated strong capabilities in generalization and performance on novel data, leading some to speculate they may contain early sparks of artificial general intelligence. However, true AGI remains an unachieved goal requiring breakthroughs beyond current techniques. LLMs have poor mathematical reasoning capabilities, inherit biases present in training data, and can hallucinate and deliver false information. In this chapter, we discuss several methods that improve the current state-of-the-art models through the use of external tools such as web browsing, better prompting techniques, scaling, using Reinforcement learning (RL) and tree-based search algorithms.