Time series data is essential in many applications of high societal relevance and the problem of efficient and effective prediction is at the heart of many recent research efforts tackling domains like energy demand prediction, weather forecasting, traffic density prediction, market predictions (to name but a few). In the recent years, multiple deep learning approaches have been proposed, each presenting different architectures/models. One universally common theme is “the more, the merrier” in terms of the (training) data. In this paper, we focus on domains in which the time series data items are bound to locations – specifically, energy demand forecasting and weather forecasting, where each data source (e.g., a power transformer units or a weather station) pertains to a particular location. While many recent works have considered the incorporation of spatio-temporal dependencies in the model to improve the effectiveness – as it turns out, the inclusion of multiple sources may have adverse impacts. We present the results of two heuristics that we developed for the purpose of determining when is it that “more is less” in such settings.