DNA methylation, an epigenetic modification that consists of the addition of a methyl group at the CpG dinucleotide sites, regulates gene expression and maintains genomic stability. Its aberrant alteration can lead to disease progression and being able to reliably predict it could be undoubtedly valuable. In recent years, classical machine learning methods and advanced transformers-based architectures have focused on predicting methylation from the genomic sequence alone; however, for the same sequence, methylation patterns can vary drastically between tissues and diseases, particularly in key genes, compromising model reliability in vivo. In this study, two existing models, iDNA-ABF and MaskDNA-PGD, which predict methylation from the DNA sequence alone, were tested on various datasets to evaluate their ability to generalize on different tissues and diseases. After being trained on their original datasets, the models were tested on independent datasets, where they achieved good accuracy despite the datasets involving different methylation and tissue types. To further assess generalization, we built a new dataset comprising methylation samples of a few marker genes collected from colorectal cancer and matched normal tissues in a cohort of patients stratified based on their level of CpG Island Methylator Phenotype (CIMP), which is characterized by an altered methylation profile. As expected, the models failed to perform well on both tumor and normal samples, due to significant differences in condition-specific methylation profiles, despite identical genomic sequences. These results suggest the need to integrate other features beyond DNA sequence to reliably predict methylation in pathological cases.