Accurate diagnosis prediction from clinical notes is critical in healthcare, particularly for cardiovascular diseases. This study develops a robust diagnostic system using a hybrid input of original and expanded clinical notes. The proposed MedBioClinicalBERT model integrates BioClinicalBERT and MedBERT to enhance medical text understanding. Tokenized clinical notes were processed through both models, and their embeddings were concatenated for classification. The model was evaluated using 5-fold and 10-fold cross-validation with metrics including average accuracy, average precision, average recall, average F1 score, average validation loss, and best validation loss. Data augmentation techniques, such as paraphrasing and numerical adjustments, improved generalization. Predicted diagnoses were mapped to ICD-10 codes using a dictionary-based approach enhanced with cosine similarity for handling unseen cases. MedBioClinicalBERT achieved 90% accuracy, outperforming individual models. Challenges included atypical note formats and limited dataset scope. This study highlights the potential of hybrid inputs for improving diagnosis prediction and suggests expanding datasets and refining mapping methods for broader applicability.