<p>With recent advances in generative AI models and their adoption by many commercial users, there has been wider discussion of the possibility that artificial general intelligence (AGI) will emerge from these systems in the near future. This discussion is muddled somewhat by conflicting conceptions of AGI and what successful trials imply. I argue for the necessity of certain social features in any system that can be ascribed general intelligence. Much of the discussion thus turns to theory of mind and attempts to model this with generative AI. These features are particularly important for a type of epistemic responsibility that has proven especially thorny for generative AI models, and I argue that the current generation of systems will not scale towards success on these fronts. While this is not an a priori dismissal of the very possibility of artificial intelligence, it implies a need to radically rethink future approaches.</p>