Conversation-based assessments (CBA), which evaluate student knowledge through interactive dialogues with artificial agents on a given topic, can help address non-effortful formative test-taking and the lack of adaptability in traditional assessment. With recent advances in Large Language Models (LLMs), this work employs evidence-centered design framework with LLM techniques to establish a multi-agent architecture for conversation-based assessments. This architecture includes four LLM agents: two student-facing agents and two behind-the-scenes agents. The two student-facing agents, the expert and peer agents, focus on asking leading questions and engaging students in providing thoughtful answers. The two behind-the-scenes agents, a formative assessor and a summative assessor, focus on analyzing and collecting evidence statements with varying granularity throughout the conversation. All agents are monitored by a non-LLM agent (Watcher), which manages the assessment flow through updated instructions to agents and turn control. We evaluated it in the context of science inquiry with secondary-level school students. Results showed that the two student-facing agents were able to differentiate diverse student responses and tried to maintain the conversational flow to be on topic. The two behind-the-scenes agents were able to analyze student answers in real-time and collect evidence for future diagnosis. The paper concludes by examining the strengths and limitations of this architecture with suggestions for future work.