The market of wireless communications is driving the technology development toward higher bandwidths and carrier frequencies. Terahertz (THz) frequencies not only open new unused bands for this purpose but also allow higher bandwidths. Yet, atmospheric attenuation above 100 GHz limit communication systems to short distances, and in many cases, non-line-of-sight links, may need to rely on specular or diffuse reflections off-walls, ceilings, or floors around us. We investigated the dielectric properties and scattering behavior of a representative collection of 50 building materials used for construction. The transmission, reflection, and scattering properties of the samples were measured using THz time-domain spectroscopy and their scattering was modelled based on the Fresnel–Rayleigh equations and the Kirchhoff theory. We observed that depending on several characteristics of the materials, the behavior could match the models very well, except for certain characteristics, such as bulk heterogeneity of the material.