With the information about the dramatic rise and success of deep learning in natural language processing and computer vision, it should be noted that one of the central innovations key to these advances is the tokenizers and encoders that encodes the unstructured data into a format digestible by the models [3]. This article attempts to provide a very detailed overview of some of the most common text encoders and to kenizers nowadays, epitomized though not limited to BERT, GPT - 2 and T5, which have gained the status as a foundation of NLP tasks. Here, we look into the details of their architectures, tokenization approaches and how they solve linguistic problems, such as rare and out-of-vocabulary words [11]. Similarly, we present different image encoder and tokenizers such as CLIP, Vision Transformer, VQ - VAE which have been instrumental for cross-modal learning and image synthesis tasks. Drawing parallels between text and image tokenization procedures, we discuss the advantages and disadvantages of each method with a special focus on multimodal scenarios where interrelation between image matching and text encoding is of paramount meaning [1].