The advent of autonomous drones promises transformative applications across many industries, necessitating the ability to plan long-range paths automatically. The purpose of this paper is to explore the synergistic effect of Markov Decision Processes (MDPs) and deep reinforcement learning (DRL) in the context of autonomous drone racing. A simulated racing track is used to demonstrate how autonomous drones can optimize path planning using the Proximal Policy Optimization algorithm (PPO). In our study, we present an agent that efficiently navigates strategic checkpoints, is adaptable, and generalizes its skills to similar situations. It demonstrates the potential of DRL and PPO to advance autonomous drone path planning, paving the way for practical implementation in a wide range of industries. With drones reshaping industries, this work contributes to the exciting future of autonomous systems. The agent covered an average distance of 19.6 m per episode, showcasing its ability to navigate the test environment effectively. In a notable feat, the agent exhibited the capability to pass through a maximum of 9 holes in the test environment without encountering any collisions. This indicates a high level of adaptability and strategic decision-making, even in the face of increased environmental complexity.