Current medical action recognition systems focus on single-modal approaches, missing valuable correlations between visual, textual, and audio information that could improve surgical workflow understanding and patient safety. We propose MedACT-CL, a tri-modal contrastive learning framework that combines surgical video, clinical text, and medical audio for real-time action recognition in clinical environments. Our approach introduces Medical Knowledge-Enhanced Contrastive Loss (MKE-CL) that incorporates clinical importance weighting from medical ontologies (UMLS, SNOMED-CT), enabling prioritized learning of clinically critical actions. The framework implements direct contrastive learning between all modality pairs to capture audio-visual correlations in surgical settings while maintaining temporal consistency through workflow-aware regularization. Experimental validation on Cholec80, M2CAI 2016, and JIGSAWS datasets shows substantial improvements, achieving 86.5% F1-score with 94ms inference latency suitable for clinical deployment. The framework shows promising performance on synthesized multi-modal samples, with validation on real tri-modal surgical data needed for clinical deployment.