CAM-GUI: A Conversational Assistant on Mobile GUI
摘要
Smartphone assistants are becoming more and more popular in our daily lives. These assistants mostly rely on the API-based Task-Oreiented Dialogue (TOD) systems, which limits the generality of these assistants, and the development of APIs would cost much labor and time. In this paper, we develop a Conversational Assistant on Mobile GUI (CAM-GUI), which can directly perform GUI operations on real devices, without the need of TOD-related backend APIs. To evaluate the performance of our assistant, we collect a dataset containing dialogues and GUI operation traces. From the experiment demonstrations and user studies, we show that the CAM-GUI reaches promising results. Our system demonstration can be downloaded from this link . You can watch our demo through this link .