Caregiver involvement in homework is a key contributor to student success, yet its interaction with intelligent tutoring systems remains underexamined. To address this gap, we conducted a technology probe study exploring how a conversational support tool might enhance caregiver assistance during remote math homework within tutoring systems. Interviews with six caregiver-student dyads conducted before and after tool use indicated that the tool primarily strengthened caregiver confidence and emotional support rather than enabling direct content assistance. Sessions were typically brief and focused, covering one to two problems. Qualitative chat log analyses highlighted design challenges, including students seeking direct answers and caregivers joining sessions with delays after SMS notifications. Despite adoption hurdles, our findings offer new evidence that tutoring systems can bolster caregiver confidence, which is a known predictor of parental engagement in education.