During the last decades, neural networks have proven to be an effective means of handling repetitive tasks that require human intelligence in industry, especially in the field of computer vision, albeit at a considerable computational cost and subsequent energy consumption. In this paper, we present a new approach for designing neural network architecture that reduces the number of model parameters, minimizes the computational cost, and improves energy efficiency without significantly impacting performance. As an example, we present iViT (Information Visual Transformer), an architecture we designed for image classification under this methodology based on “pseudo-embeddings” and a MoE (Mixture of Experts) instead of traditional patch embedding.