Collaborative Commonsense AI

Collaboration is one of the core human principles and abilities for solving difficult problems and scaling complexity. Under an assumption that the strong sides of AI are different than those of people, and encouraged by anecdotal success stories like “centaur chess” and by large models communicating in a natural language like ChatGPT, the question emerges: (how) can AI work in synergy with humans? What capabilities, interfaces, and models does AI need to have to be collaborative with people and make a potential impact through collaboration? While it may be difficult to draw a strong distinction between collaborative and non-collaborative mechanisms, several intuitive directions emerge from the literature. Collaborative AI needs to be able to build an internal model of situations, including the relevant actors and objects, their attributes, states, relationships, and affordances. It needs to have an internal model of itself, namely, its goals, plans, and history of prior experiences. It must have Theory-of-Mind, i.e., an evolving representation of the beliefs, goals, and other mental attributes of other agents in a situation. And it must apply such collaborative skills within multimodal interactions, involving language, vision, and planning. We review state-of-the-art methods and findings for such collaborative mechanisms and conclude with a discussion of open challenges and lessons learned.