Improving Large Language Models in Multi-party Conversations Through Role-Playing

Large Language Models (LLMs) have demonstrated impressive performance in a wide range of natural language processing tasks and have been successfully applied in two-party dialogue systems. However, the capabilities of LLMs in Multi-Party Conversations (MPCs), which are prevalent in real-world scenarios and find many applications in dialogue systems, are largely underexplored. In this paper, we propose an MPC generation framework named Role-Playing Multi-Party Conversation (RPMPC), which utilizes an LLM to generate a sequence of multi-party dialogue among multiple participants. RPMPC leverages a novel Role-Playing Utterance Prompting (RPUP) technique that constructs a list of prompt messages by separating previous utterances of a selected speaker and those of other speakers into alternating “assistant” and “user” messages. Such prompting technique helps an LLM maintain consistency and identity in MPCs. In addition, to balance coherence and diversity of the generated dialogue, we incorporate a hybrid turn-taking (i.e., next speaker selection) strategy into RPMPC by probabilistically mixing an LLM based turn-taking strategy and a random strategy. Experimental results on real-world datasets show the effectiveness of the proposed RPMPC framework in MPC generation.