With the rapid expansion of social networks, the widespread propagation of rumors poses a significant threat to information security and societal stability. There are some challenges in current rumor detection work. Firstly, social media data’s disorderly and chaotic nature presents substantial obstacles to extracting deep semantic features. Secondly, the diverse and complex user interaction relationships in social networks make it difficult to mine potential interaction features. To address these challenges, this paper introduces the Context-Aware Social Interaction Rumor Detection Network (CASINet), designed to explore contextual semantic features and social network user interaction characteristics, alongside the efficient fusion of heterogeneous information. The framework comprises three core components: The Contextual Semantic Interaction Module, which includes a context-aware semantic encoder and multi-level feature extractor for mining deep contextual semantic features; The Social Network User Interaction Module, which constructs a heterogeneous graph between users and tweets to capture the latent interaction relationships; and the Heterogeneous Feature Fusion Module, which enhances the model’s generalization ability by automatically aligning and deeply integrating heterogeneous features. Experimental results validate the substantial enhancements achieved by our model in early rumor detection, providing a potent solution to the pervasive issue of rumor propagation in social networks.