GeoAI is increasingly deployed in the humanitarian mapping of vulnerable communities in the Global South. Specifically in the mapping of ‘slums’ and informal settlements, as well as for disaster risk management (DRM). GeoAI presents several opportunities for improving the visibility of the communities, revealing the precarity they experience living in such areas. But it is also important to consider the more ethical and existential consequences that can arise from these mapping initiatives. This chapter will begin by tracing the development of cartography and its socio-political influence, drawing on insights from critical cartography and critical geography. These insights help emphasise the fact that maps and map-making are socially and politically embedded and not purely technical or objective in representing the world. This will also be a pertinent and instructive starting point for taking a more critical stance on GeoAI. This chapter will present an ethical examination of the humanitarian use of AI, addressing issues such as accountability, bias, political representation, and the potential inequalities in the creation and distribution of geospatial knowledge in the new wave of ‘digital humanitarianism’ (Fejerskov et al., Disasters 48(2):e12609, 2024).