Histopathological image recognition requires efficient feature extraction and dimensionality reduction to manage the complexity of the scans. For this, we propose a hybrid models that combine TinyViT with the self-supervised learning capabilities of DINOv1 and dimensionality reduction techniques, which achieve higher accuracy with improved computational efficiency outperforming DINOv1 architectures. To enhance performance, feature reduction techniques, such as PCA and NCA, are employed to reduce feature sizes while minimizing accuracy loss and retaining critical information. Although DINOv1 exhibits state-of-the-art accuracy in general computer vision tasks, its performance on medical images at the researched magnification is limited. In contrast, TinyVIT-based models offer a balanced solution to efficiently process large histopathology scans with improved accuracy and reduced computational requirements, as our results show.