Nowadays, there is a huge growth in online services as well as the quantity of varied data that is being transferred across the internet which requires a change from traditional networks to a form of networks that can adapt to the exponential demand of the end users. Traditional networks fall short in providing the essential requirements of end users such as quality of service and high efficiency because it cannot go on the way that technology proceed. Software Defined Networking (SDN) provides a solution to these issues because of the nature of its architecture which includes network functionalities in an open and programmable manner. Nevertheless, this form of architecture also leads to various security challenges and including the denial-of-service (DoS) attack, and to detect those attacks various solutions have been proposed, many of them used Machine Learning (ML) and Deep Learning (DL) algorithms to not only detect but also predict potential attacks. These solutions can detect and mitigate different types of attacks within seconds and prevent network resources from being exhausted. Hence, this paper we present a review of the existing literature on the application of machine learning and deep learning algorithms for software-defined networks security.