<p>This study investigates the behavior and characteristics of word embeddings through a series of experiments, focusing on models such as Word2Vec, FastText, and BERT. First, we analyze word embeddings under extreme simulated conditions by creating a data corpus without any interrelation of words and compare the resulting embeddings with randomly initialized vectors. This experiment reveals the tendency of word embedding models to create apparent word relationships even in the absence of real contextual connections. Second, we examine the evolution of word similarity with respect to increasing frequency of co-occurrence, demonstrating how embedding similarity changes as word co-occurrence frequency increases and identifying potential saturation effects. Third, we employ machine learning models-including random forest and XGBoost—to predict word similarity based on various statistical and lexical characteristics derived from real-world data. We identify influential factors such as co-occurrence frequency, context correlation, word frequency, and human-judged similarity scores, which significantly contribute to the prediction of word similarity. The prediction of semantic similarity was formulated as both regression and classification tasks, achieving high accuracy, with the random forest regression model yielding a Pearson correlation of 0.84 (p !' 5e-145). Interpretability methods such as feature importance, partial dependence plots (PDP), and local interpretable model-agnostic explanations (LIME) were used to reveal important relationships and provide insights into the factors influencing embedding similarity. Our findings contribute to a deeper understanding of how word embeddings capture semantic similarity and the factors that influence this process, offering valuable implications for the development of more transparent natural language processing models.</p>