Critical infrastructure increasingly relies on embedded systems, making them particularly vulnerable to cyber attacks due to their complexity and interconnectivity. Unlike general-purpose systems, embedded systems need specialized security solutions tailored to their unique vulnerabilities. Accurate classification of embedded system vulnerabilities is essential for targeted analysis and mitigation. Traditional methods using pre-trained embeddings like Word2Vec, GloVe, and FastText often struggle with Out-of-Vocabulary (OOV) words, reducing their effectiveness. We address this with a novel ensemble embedding technique that combines multiple pre-trained embeddings, enhancing the classification of embedded system vulnerabilities. Our BiLSTM-based model, tested on datasets such as NVD and CNNVD, achieved 82.61% accuracy on unseen data, outperforming traditional embeddings.