Hyperdimensional Computing (HDC) is a powerful technique for dynamic and low-resource learning, an ideal candidate for data-agnostic centralized communication pipelines in an Artificial General Intelligence (AGI). When inputs can be effectively encoded into hypervectors, learning tasks are reduced to rapidly computed algebraic statements on large vectors. However, the act of encoding - or even producing codes via a separate Machine Learning model - remains one of the biggest bottlenecks in the process. Oftentimes this reduces to choosing somewhere between high accuracy or high speed as optimizations. In this paper, we introduce the notion of Hyperdimensional Propagation of Error (HyPE) - the HDC framework’s answer to back-propagation in Neural Networks - and demonstrate its potential in such settings. Namely, we introduce a layered HDC architecture that self-organizes itself into longer and longer hypervectors according to a method akin to Generalized Expectation-Maximization (GEM) algorithms. This allows low-order encodings to be projected to longer sizes gradually and thus produce powerful encodings with less resources and smaller vectors. We demonstrate the potential of this on Fashion-MNIST and discuss how the idea of “HyPE-ing” encodings could be used in other contexts.