Governments around the world are aiming for net-zero carbon emissions by 2050. To drive broader decarbonization efforts, the building industry is challenged by the mission to reduce embodied carbon emissions, which stem from building materials and systems. Most efforts in buildings are focused on operational energy, but up to 80% of a building’s emissions occur before use and occupancy, from extraction to construction phases. These emissions are irreversible and contribute around 11% of global carbon emissions. This study emphasizes the need to prioritize the decarbonization of conventional building materials during production using low carbon constituents to mitigate associated environmental impact. Quantitative data analysis and reviews have identified alternative low-carbon options for possible application in mix design. Examples include substituting general use (GU) Portland cement with Portland limestone cement, using supplementary cementitious materials (SCMs) to reduce cement content, and utilizing low-carbon concrete masonry units. Achieving net-zero embodied carbon requires promoting circular bio-based products and reducing conventional materials’ carbon throughout their life cycle. At material level analysis, this is possible not only by promoting the use of circular bio-based materials but also by purposefully reducing the embodied carbon of conventional building materials throughout their life cycle, as well as communicating the best practices as lessons learnt to promote broader adoption by the architecture, construction and engineering (ACE) industry. However, collaboration among designers, contractors, and manufacturers is essential. This study provides a preliminary pathway to overall decarbonization efforts, understanding that conventional, existing building materials will play a significant role in attaining the net-zero commitments of the future built environment.