Pepper is one of the most important vegetable crops in the world. The use of male sterile lines for hybrid seed production is considered to be an ideal approach to exploiting heterosis in pepper. There are two main types of male sterility (MS) in pepper—cytoplasmic male sterility (CMS) and genic male sterility (GMS)—and each has its advantages and disadvantages when used for hybrid seed production. In this chapter, we have systematically reviewed and summarized the sources of pepper GMS and CMS, the inheritance of pepper MS and fertility restoration, and the genetic mapping and identification of important candidate genes for pepper GMS, CMS, and restorer-of-fertility (Rf) genes. Finally, we look forward to future research directions and considerations in this field. We hope that this review will serve as a reference for research on pepper MS and fertility restoration.