A large amount of the building sector carbon footprint is caused by the manufacturing of cement, a vital component of concrete. New techniques for its production are continuously being developed to lessen its environmental impact. Using natural waste materials to partially replace cement is one such technique. This work primary goal is to correlate the used quarry waste mineral composition to the produced concrete mechanical performance at 28-day age: X-ray diffraction (XRD) analysis was used to investigate the quarry material components. It is an easy accessible by-product of stone crushing processes. It was exploited in a partial substitution procedure of the classical cement binder. Concrete samples were made using 2 different waste quarry powders P1 and P2 respectively at 2 cement substitution rates: 10% and 20% of the cement weight. P1 is a limestone based material whereas P2 powder is a clay based one. Both powders were baked at 650 °C during 6 h before use. Sodium silicate Na2SiO3 and sodium bicarbonate NaHCO3 alkaline activators were respectively added to the concrete mix at a 20% dosage of the waste amount. Experimental test results illustrate that the substituting powders have positive effects on the 28-day compressive strength (Rc28). The Rc28 growth rate obtained in the case of P1 waste ranged between 11.6% and 24.3% compared to the reference sample (with no cement substitution). For P2 clay based waste, the compressive strength increase rate reached 6.2% at a substitution rate of 20% when the activator was Sodium Silicate. In order to understand how the waste powders have helped to increase concrete compressive strength, the XRD analysis was adopted. It seems that calcium, Silica and Zinc presence in the quarry waste have helped to enhance the cement hydration process resulting in additional hydrates that reinforce concrete structure. This study emphasizes the possibility of recycling quarry waste materials in manufacturing low environmental impact concrete without sacrificing its mechanical performance. The findings show that substituting up to 20% of the cement binder could preserve the compressive strength.