Robust Commonsense AI

The emergence of (large) language models has resulted in a paradigm shift in terms of developing general-purpose models that can contribute to a variety of tasks and domains. For the first time in the history of AI, the community of researchers has access to pre-trained models that can perform repeated inference across many unanticipated natural language tasks, from text classification through question answering to summarization. Moreover, these models can be tuned further to fit the needs of the downstream task more directly by using adequate training data, thus enhancing the model performance on target tasks. While the generalizability of language models is clear and can be witnessed by experts and non-experts alike thanks to public APIs like ChatGPT, their robustness has been repeatedly shown to be fragile. Language models struggle with adequately weighing minor variations in the inputs (e.g., in negation or perturbation probes), they may be unable to perform well in novel settings (e.g., in domain-specific applications), and their predictions may be inconsistent across different phrasing of the same input (e.g., in information extraction settings). This chapter discusses these challenges in more length and provides a summary of directions for designing robust AI that leverages the power of LLMs in combination with data augmentation with other models or graphs, with novel methods for language model adaptation such as in-context learning and prefix-tuning, with neuro-symbolic integration with code and logic interpreters, and with analogical principles. This chapter discusses representative methods from each of these research directions in terms of their motivation, design, results, strengths, and weaknesses. The chapter concludes with a summary of lessons learned and open challenges that can be pursued to develop even more robust AI in the future.