Due to the growing world population, the production of food of animal origin will increase in the coming years. Currently, worldwide up to 40% of all arable land and more than 30% of cereal crop production are related to the production of animal feeds. Nowadays, livestock nutrition is based on a wide range of expertise, including animal physiology, feed science and technology, agricultural science, biochemistry, and animal health. The aim is to keep animals healthy and maximize their performance while minimizing environmental impact. Food production must be optimized in terms of environmental impacts and food security. Thus, changes in food production inevitably affect the feed production chain. By-products from different origins are widely used in animal nutrition. Many of them are not suitable for human nutrition, which means their use in animal nutrition may reduce the feed-food competition. Additionally, reducing food waste, increasing recycling, and enhancing ex-food management are tools to reduce the environmental impact of feed and livestock production and to develop them to a higher circularity. Balanced for the major nutrients, many of these products have potential as animal feed. The substitution of traditional proven feedstuffs with by-products could be an interesting prospect for animal nutrition.