Neural networks are now widely used in natural language processing (NLP) tasks, with BERT being one of the most common architectures. These models are frequently deployed on low-memory devices like mobile phones where response time is critical, leading to the development of various compression techniques including distillation, pruning, and quantization. We propose a genetic algorithm-based approach that effectively combines these methods to optimize both model size and inference time while maintaining quality. Our evaluation examines the algorithm’s performance across different hyperparameter configurations and compares the best-performing compressed BERT variant against other commonly used compressed models like TinyBERT. The analysis provides practical recommendations for hyperparameter selection based on specific machine learning task requirements.