Cross-lingual semantic similarity analysis can be a valuable tool for evaluating the quality of educational material adaptation and cultural fit across cultures and languages. This study assesses semantic and cultural quality from an Adolescent Literacy program (Word Generation, HARVARD & SERP) adapted into Brazilian Portuguese. Using Sentence-BERT MiniLM, and XLM-RoBERTa, we evaluate semantic similarity in two cases: English text vs. Portuguese version and English text vs. back-translated English. Back-translation yields higher similarity due to literal translation, while cultural adaptation introduces linguistic shifts with slightly lower, yet high, similarity scores. Multilingual BERT offers stable alignment; Sentence-BERT MiniLM finds sentence-level similarities; XLM-RoBERTa captures detailed token-level shifts. Findings highlight the translation fidelity and cultural relevance trade-off and emphasize the need for automated and expert validation. The study suggests an AI-assisted validation framework for educational material adaptation, advocating for integrating qualitative and automated measures.