Hydrogen is emerging as a sustainable solution to the challenges associated with traditional energy sources such as greenhouse gas emissions and climate change. It serves as an energy carrier to convert and store excess energy produced by renewable sources until demand rises. Comparatively, underground storage options are more viable for large-scale hydrogen storage, with depleted gas reservoirs offering significant advantages such as higher storage capacity, prior experience in gas storage, existing infrastructure, geological feasibility, and lower capital and operational costs. Caprock, an impermeable layer overlying the porous reservoir rock, is a crucial component in these subsurface reservoirs to securely entrap injected hydrogen and prevent its migration into upper layers. However, the caprock integrity is greatly affected by the high-pressure hydrogen injection, geochemical interactions in the hydrogen-rock-brine system, and cyclic operations. These factors may lead to multiple crack developments in the caprock, creating new pathways for hydrogen leakage. Critical crack propagation is one of these multiple cracking mechanisms controlled by the stresses applied to the rock. High-pressure hydrogen injection poses a potential risk of caprock failure under critical crack propagation. This paper presents a systematic review of caprock integrity in the underground hydrogen storage context, with a specific focus on critical crack propagation, using a scientometric approach. The findings suggest that while significant attention has been given to geochemical interactions and hydrogen diffusion through the caprock, critical crack propagation in the caprock remains underexplored in the existing literature. Thus, this paper further presents a suitable experimental methodology to determine critical crack propagation in the caprock during underground hydrogen storage by using rock fracture toughness as a quantitative parameter.