Reproducibility is a cornerstone of scientific progress, allowing researchers to validate findings and build upon previous work. While reproducibility has been an important issue in traditional recommender systems, the rise of Large Language Model (LLM)-based recommendation systems introduces new challenges, particularly in top-N recommendation tasks. In this study, we investigate the reproducibility of state-of-the-art LLM-based recommendation systems. We categorize key factors affecting the reproducibility of recommendation performance into four groups: code, data, methodological details, and evaluation. Our findings highlight significant performance fluctuations based on these factors, emphasizing the need for these factors to be clearly documented and considered during evaluations. To enhance reproducibility, we propose LLMReClarify, a comprehensive set of guidelines adapted from the NeurIPS reproducibility checklist, tailored specifically for LLM-based recommendation systems.