This paper examines the quality of Generative Artificial Intelligence responses to open-ended questions in the context of computer science and evaluates the reliability of embedding-based similarity models in approximating human assessment. A dataset of 800 answers was collected from ten GenAI models across eight domain-specific questions. Each answer was graded by a human annotator, and four distinct models (SBERT, GPT, GPT-4o, Cohere) were used to calculate semantic similarity between GenAI answers and reference materials. The analysis used Quadratic Weighted Kappa to assess agreement between models estimated grade and human annotation. Results showed that only GPT-4o demonstrated moderate positive correlation with human grades and meaningful variation across grade levels, particularly when two problematic questions were excluded from the analysis. Other embedding-based models failed to align with human annotations, often misclassifying grades and failing to identify strong or weak quality answers. These findings challenge the assumption that semantic similarity can serve as an alternative for human assessment in evaluating open-ended questions. While GPT-4o showed some potential under constrained conditions, all models exhibited significant limitations.