The increasing amount of plastic waste disposal and the overexploitation of natural aggregates during concrete production highlights the potential of integrating plastic waste into concrete as a viable solution. This study investigates the physical properties and the durability of concrete containing plastic granules as a partial replacement for natural aggregates. The studied concrete mixtures were manufactured with 20% and 40% replacement of natural aggregates with recycled polystyrene (PS) granules. The compressive and flexural strength, thermal conductivity, and capillary water absorption were measured and compared to the conventional concrete properties. In addition, the properties were assessed before and after an accelerated laboratory ageing through exposure to rapid freezing and thawing in a climatic chamber. The results showed that increasing the plastic granules content decreased mechanical strength and thermal conductivity while water absorption increased. Findings highlight how plastic granules incorporation affects the mechanical performance and durability of the newly investigated composite material, particularly under temperature fluctuations and high humidity. These properties are crucial for developing more sustainable concrete alternatives, especially for applications in conditions with significant temperature variations and moisture exposure. Further research is, however, needed for a more precise understanding of the concrete with PS granule’s behaviour.