Lung cancer is a common kind of cancer determined through unregulated proliferation of cells in pulmonary tissues. Signs may include chest pain, persistent cough, and shortness of breath. Early identification by screening procedures is crucial for prompt treatment and better results. In this research, authors aim to conduct a systematic review of machine learning methods for lung cancer detection. The latest advancements in machine learning algorithms and their significant trends, challenges and future directions were discussed. We used significant keywords for searching the articles through the internet and the PRISMA model was employed to summarize the final selection of articles. Authors discussed the characteristics of the standard lung cancer dataset, which is widely utilized to train machine learning models. Authors also evaluated the various types of deep learning models for detecting lung cancer disease. Across investigations, machine learning algorithms performed better than the conventional deep learning approaches in detecting lung cancer in CT and MRI images. It is demonstrated by the potential of machine learning models to represent complicated spatial connections and automatically extract discriminative characteristics from medical images. Machine learning algorithms establish significant potential for lung cancer detection and diagnosis. However, further studies will be required to assess all models and handle various challenges. This study provides valuable insights for summarizing developments and identifying possibilities for improving the characteristics of machine and deep learning algorithms in medical image analysis.