Large language models (LLMs) are artificial intelligence (AI) tools that can generate answers to a variety of student questions. While these tools have the potential to provide personalized instruction, there are also concerns that students will over-rely on them to obtain answers without trying to learn the target concepts. Since LLMs are relatively new, research is needed on their impact on student learning and ways to scaffold learning. We conducted an experimental study (N = 101) in which students solved code-tracing problems. We manipulated (1) the presence of prompts encouraging active and constructive engagement, and (2) the type of instructional resource available, either ChatGPT or traditional materials. The prompts significantly increased learning across both instructional resource contexts; there was no significant difference between the ChatGPT and traditional-materials groups. To gain insight into how students used ChatGPT, we used qualitative methods to analyze their interactions with the tool.