Efficiently learning visual representations of items is vital for large-scale fashion recommendations in e-commerce. In this article we compare several pretrained efficient backbone architectures, both in the convolutional neural network (CNN) and in the vision transformer (ViT) family. We describe challenges in e-commerce vision applications at scale and highlight methods to efficiently train, evaluate, and serve visual representations. We present ablation studies that evaluate visual representations in several downstream tasks. To this end, we present a novel multilingual text-to-image generative offline evaluation method for visually similar fashion recommendation systems. Finally, we include online results from machine learning systems deployed in production on a large-scale e-commerce platform.