As a significant engineering means against rockfall disasters, the rockfall galleries could provide protection for the safety of transportation, life, and property in mountainous areas. Considering the construction convenience and material accessibility, a buffering cushion layer consisting of gravel is often laid on top of the rockfall gallery in practice to reduce the impact of falling boulders, hence improving the impact resistance of the gallery. Therefore, the buffering performance of the gravel cushion requires to be comprehensively researched and further explored. Based on the random generation method of polyhedrons, the present study establishes a mesoscale finite element (FE) model of gravel cushion, which is validated by existing drop-weight impact tests. By conducting numerical simulation with the proposed FE model, the buffering performance of the gravel cushion under different impact energies is discussed, and the distribution of impact force on the bottom slab, i.e., the roof of the gallery, after passing through the cushion is explored. The proposed mesoscale FE model of gravel cushion could exhibit the discreteness and irregularity of the material properties and impact outcome of gravel cushion under rockfall impact in actual engineering scenarios. The results of the present study can extend the impact scenarios of relevant tests, specifically for drop-weights with large mass and impact velocity, and correspondingly give suggestions and references for the practical design of rockfall galleries.