In this paper, we introduce RowFormer, a novel transformer architecture that treats an image as a collection of sentences rather than a single one. This approach addresses challenges related to modifying class tokens and projection mechanisms. RowFormer combines the strengths of convolutional neural networks and vision transformers by integrating spatial convolution, as seen in Convolutional Vision Transformer and in Compact Convolutional Transformer. However, in this work, we introduce a novel transverse multi-class token design for vision transformers, contrasting with the default single-token approach. This approach would be well suited to scenes, multi-object images, and single-object images as well. Our experiments demonstrate that RowFormer achieves higher efficiency across multiple classification benchmarks compared to existing models. This improvement stems from our hypothesis that images should not be interpreted as continuous, paragraph-like representations but as structured sequences of rows, each providing its own information. By leveraging this insight, RowFormer enhances visual data processing, enabling more effective image classification and understanding in complex tasks.