This study analyzes AI model evaluation by examining (1) the effect of temperature on model outputs and (2) a comparative study between reasoning and non-reasoning AI models. Based on a previous study (Schmidt et al. in AI-enhanced QOC analysis: a framework for transparent and insightful decision-making. Springer, pp 415–429, 2024) [1], in which we performed a human-conducted QOC analysis followed by an evaluation using OpenAI models, this work extends the assessment by systematically analyzing different temperature settings in non-reasoning models and comparing reasoning and non-reasoning AI models. The analysis includes reasoning and non-reasoning models, where temperature settings are varied incrementally for non-reasoning models, while reasoning models are assessed based on different reasoning effort configurations. Statistical comparisons identify the optimal settings for each model type. The results indicate that temperatures around 0.9 in non-reasoning models achieve a good balance between stability and diversity, which aligns best with human evaluations, while a lower reasoning effort is beneficial in the o3-mini model.