Supervised X-ray threat detection methods are usually impractical due to their reliance on large volumes of labeled data. Meanwhile, unsupervised approaches often fall short in distinguishing threats from benign objects due to the intricate grayscale textures and significant object overlap present in X-ray scans. To tackle these challenges, we propose X-ThreatDet, a Self-Supervised framework designed for X-ray threat detection without requiring manual annotations. Our approach incorporates Cutler for proposal region extraction, facilitating zero-shot object proposal generation. Additionally, we introduce contrastive multi-modal clustering, image and text encoders, to categorize detected proposals into threat and non-threat groups. Furthermore, a self-supervised knowledge distillation module built on a teacher-student model enhances multi-scale feature learning by refining representations from both global and local image crops. We extensively evaluated X-ThreatDet on two benchmark datasets, PIDray and CLCXray, where it achieved state-of-the-art (SOTA) performance.