<p>One of the most life threatening diseases among women of all ages in the world is the cervical cancer and early diagnosing and proper diagnosis is very critical in the successful treatment of the cancer. However, the medical information is extremely heterogeneous, and it cannot be properly identified since the medical information may be introduced in unstructured form, such as medical images and clinical text reports. The traditional diagnostic models tend to take either the image or textual data as uncomplimentary and this limitation, makes them limited to giving rise to the opposite relationships between the two modes. To address this weakness, this paper introduces a novel multimodal architecture called Dual-Input Deep Canonical Fusion Network (DIC-FNet), where it is assumed that both medical images and clinic text are the inputs to the same network. To extract the information, the framework draws discriminative visual characteristics (Pap smear and colposcopy) and semantic characteristics (patient history, clinical reports and diagnostic notes) to extract the information. Fusion is done using canonical correlation between these heterogeneous features forming a type of features that captures cross-modal relationships that may be used to spur improved diagnostic performance. Besides, a new ChaKho optimizer is offered as a Fusion Weight Estimator to increase the image text interaction. The results of experimental tests demonstrate that DIC-FNet with ChaKho optimization delivers better results than all current methods. The system achieves 99.00% overall accuracy with 98.74% precision 98.36% recall and 98.55% F1-score which surpasses the performance of existing multimodal baseline systems. The framework achieves its classification capability through dependable output which it produces with 98.87% specificity 99.21% AUC and 0.968 MCC results while maintaining a 1.00% error rate.</p>