Vector databases are a critical component in modern system infrastructures. In this study, we discuss the principles behind vector database management systems, with a focus on their features, the concept of vector embeddings, and similarity search mechanisms. Furthermore, we examine the synergies between vector databases and language models, which rely on vector embeddings for semantic search and retrieval-augmented generation. We also discuss the challenges arising from the integration of language models with vector databases. Through this discussion, we aim to provide early-stage researchers with an overview of the integration of vector databases and language models.