Whereas human communication is inherently incremental, this is mostly not the case for current explainable AI (XAI) approaches. Incrementality in human–human interaction (HHI) serves to achieve smooth and fast interaction. This makes it possible, in the case of explanations, to continuously monitor an explainee’s understanding and identify and locate misunderstandings very precisely in order to scaffold the interaction partner online. It also allows the chunking of complex meanings into smaller units that are easier to handle and remember. The two processes together allow human interlocutors to develop a new quality of interaction by relying on and referring to previous jointly established routines and concepts, thereby facilitating further interaction and achieving a progressivity toward a joint explanatory goal. This is currently lacking in XAI systems. Whereas some forms of incrementality have been implemented in human–computer interaction (HCI) systems, it is still largely, with few exceptions, unaccounted for in XAI research. We present some incremental systems and discuss what technological advances are needed to achieve similar efficiency in explaining processes to that in human interaction.