Objective: This study aims to explore the application of Chain of Thought (CoT) reasoning in automating ICD coding, specifically focusing on lymphoma cases. By leveraging large language models (LLMs) and CoT prompting, the research aims to improve the accuracy and interpretability of automated ICD coding in healthcare. Methods: We optimized CoT prompting for Qwen 2.5:14b to guide the model in reasoning through the diagnostic process in a step-by-step manner. There are three main components for our pipeline. This framework was then evaluated on multiple LLMs, including Qwen 2.5, Llama 3.1, and Gemma 2, using a clinical lymphoma dataset for automated ICD-10 and ICD-O coding tasks. The models were assessed on their ability to extract relevant diagnoses from clinical text and map them to appropriate ICD codes. Results: The results indicated that CoT reasoning improves the interpretability and accuracy of ICD coding. However, the generalizability of CoT-based prompt design is not guaranteed for all the LLMs. Prompts optimized for one model do not consistently transfer to others, highlighting the importance of customizing prompts for each specific model. Conclusion: Our findings suggest that CoT reasoning can enhance automated ICD coding when tailored to the characteristics of each model. Future work should focus on optimizing CoT prompting techniques for different models to enhance generalizability and improve the scalability of automated coding systems in healthcare.