We introduce the use of machine learning in the cryptanalysis of code-based cryptography. Our focus is on distinguishing problems related to the security of NIST round-4 McEliece-like cryptosystems, particularly for Goppa codes used in ClassicMcEliece and Quasi-Cyclic Moderate Density Parity-Check (QC-MDPC) codes used in BIKE. We present DeepDistinguisher, a new algorithm that trains a transformer to distinguish structured codes from random linear codes. The results show that the new distinguisher achieves high accuracy in distinguishing Goppa codes, suggesting that their structure may be more recognizable by AI models. Our approach outperforms traditional attacks for distinguishing Goppa codes in certain settings and generalizes to longer code lengths without further training using a puncturing technique. We also present the first distinguishing results for MDPC and QC-MDPC codes.