Thermal loading and unloading of packed beds causes individual particles to expand and contract, which leads to a build-up of stresses and strains inside the filling material of a thermal energy storage. As a result of this, material degradation of aggregates or even structural failure of the storage can occur after a large number of loading-unloading cycles. Furthermore, thermal ratcheting or accumulation of permanent deformation can take place, a phenomenon especially prominent when there is a discrepancy between the thermal coefficients of the storage wall and the granular material. This study focuses on a numerical investigation of the storage wall coefficient of thermal expansion using a hypoplasticity finite element model. The proposed approach employs the standard hypoplasticity constitutive model as well as the intergranular strain extension of the standard model to capture the behavior of the granular material. The storage wall is assumed to be rigid, but it expands and contracts due to the heating and cooling of the system. It is shown how changes in the storage wall expansion coefficient affects the distribution of stresses and strains along with the compaction evolution of the filling material inside the thermal energy storage. In this investigation, both the single loading and the cyclic loading cases are studied.