The great biodiversity found throughout Middle and South America is due to past and present abiotic and biotic factors that have shaped the richness and abundance of animals over millennia. One of the main drivers during the last 10 million years has been the Great American Biotic Interchange (GABI), which was facilitated by the formation of the Isthmus of Panama and substantially changed the taxonomic composition of Central and South American faunas. Another major factor has been great climate instability, with glacial and interglacial periods occurring at different intervals, starting at the end of the Pliocene and continuing throughout the Pleistocene. We review the historical context of Neotropical mammals by describing the groups that inhabited South America prior to the GABI, and we use two well-sampled Middle Miocene faunas as examples of typical pre-GABI mammal communities. We review the chronology of the GABI and the changes in faunas and habitats that transpired from the Late Miocene through the Late Pleistocene and discuss possible causes for the megafaunal extinction that eliminated most large mammals from South American ecosystems at the end of the Quaternary.