Representation in a broad sense is how a particular object is presented in the space of internal states of the perceiving subject. Inspired by the works of V.A. Lefebvre, we call the formation and use of representations “reflection in a broad sense”. We study representations in extremely simple model objects – recurrent neural networks of small size (30 neurons). One of the simplest tasks that requires the presence of representations for solution is responding to fixed time series of stimuli, in which the neural network should recognize the series feeding to the input and thereby predict the next stimulus it will receive. The representation of a series is considered as a dynamic pattern of neural activity distinguishable from representations of other series. This pattern can be decoded, i.e., based on its type, it is possible to define the certain series fed to the input of the neural network. In this paper, we check whether there are differences in decoding of homogeneous and heterogeneous recurrent neural networks’ neural activity, considering decoding using feedforward networks and the K-nearest neighbors method. Configurations with temporal heterogeneity (DTRNN) demonstrate differences from homogeneous ones, while configurations with functional heterogeneity (RefNet) behave similarly to homogeneous configurations. This result is consistent with the literature data on the significance of temporal coding obtained on more complex systems. In the future, we plan to continue working models that have temporal heterogeneity in order to formulate a mathematically and neurobiologically substantiated measure of the neural activity decodability.