This study highlights key challenges in deploying transformer-based models, particularly their high computational demands and limited feasibility in low-resource environments. Pruning, both structured and unstructured, has emerged as a promising solution to reduce model complexity while maintaining performance. However, this approach presents unresolved issues, such as the trade-off between sparsity and accuracy, lack of generalizability across tasks, high pruning costs, and limited interpretability. This article contributes by emphasizing empirical validation, scalability across large models, and the use of domain-specific benchmarks. It also addresses hardware constraints and explores the role of interpretability in practical pruning implementations.