Primordial black hole formation requires high-density regions in the (random) density field filling the primordial universe. While only the largest (and so rarest) overdensities collapse to form primordial black holes, the rest cause large anisotropic stresses, which are the source of gravitational waves. We provide an overview of the theoretical aspects of the gravitational wave backgrounds associated with primordial black holes from large primordial fluctuations. We consider gravitational wave backgrounds associated with primordial black hole formation, primordial black hole reheating, and unresolved primordial black hole binaries. We present several graphical summaries and illustrations for the busy reader.