Introduction <p>Large language models, increasingly utilized across various fields including academia, are being explored for the evaluation of materials and methods sections in scientific publications.</p> Methods <p>A comprehensive, model-derived scoring system was developed using ChatGPT 5.0 for assessing the scientific methodology of research articles. In the study, the methods sections of 20 randomly selected articles from two reputable journals were evaluated by both ChatGPT 5.0 and two human experts.</p> Results <p>In general, similarity between ChatGPT and expert evaluations was observed except for certain criteria. The average overall given by ChatGPT was 45.5 (5.75), compared to 48.0 (4.00) in the expert assessments, and this difference was not statistically significant (<i>p</i> = 0.13). However, in the field of "data collection methods." experts gave significantly higher scores than ChatGPT (<i>p</i> = 0.01). Consistency analyses showed that the agreement between the ChatGPT and expert assessments was generally low. Friedman's test showed that there was no significant difference in ChatGPT ratings (<i>p</i> = 0.050), indicating borderline significance, but there was significant internal variability in human expert ratings (<i>p</i> = 0.04).</p> Conclusion <p>The findings of the study show that ChatGPT is able to produce results close to those of experts in methodological evaluations and perform consistently on certain criteria. However, it has limitations such as low agreement with human experts in areas that require contextual understanding, such as ethics, and a tendency to systematically give lower scores. AI-supported tools should be integrated in a way that supports expert judgments and complies with the principles of transparent use.</p>