Hate speech detection identifies harmful language that promotes discrimination, violence, or prejudice. The spread of hate speech on social media has a detrimental impact on individuals and society, leading to emotional distress, inciting violence, and reinforcing harmful stereotypes and discrimination. This paper addresses the challenge of automatically detecting hate speech in English-Hindi code-mixed text, a task complicated by linguistic complexity and cultural nuances that have received limited attention. To tackle this issue, we develop two bilingual Hindi-English BERT-based large language models. We then fine-tune these models for hate speech detection in English-Hindi code-mixed text and evaluate their performance against several massively multilingual language models.