Citizens affected by natural hazards share data on social media (SM) that can help notify first responders, keep them informed and contribute to situational awareness. Previous research focused on analysing this content, such as SM user-generated data, that is used for sentiment mining. The present work will lay out our study design building up on previous research, yet shifting the focus from citizens to first responders, such as Civil Protection Authorities (CPAs). In this upcoming study, we are going to present to CPAs a tool that has proven successful, i.e. providing insights ahead in time of the authorities, after the 2020 earthquake in Groningen, Netherlands. This tool is equipped with AI-functions that can contribute to the situational awareness during earthquakes. One is an NLP-based sentiment analysis model that analyses data based on positive and negative sentiment, and negative sub-sentiment (hatefulness, anger, fear, confusion, sadness). This functionality is co-developed and improved during a H2020 project, where CPAs have tested the creation of a dashboard for the 2023 Türkiye–Syria earthquakes. The sentiment analysis identified the spread of fake information online and the impacts of psychological distress. With the other AI-function, an entity extraction model, key facts were aggregated and plotted on a timeline, showing the development and impact of the earthquakes. In our upcoming study, we are going to conduct a survey with CPAs concerned with natural and anthropogenic hazards such as the Türkiye–Syria earthquake. We are going to measure the CPAs’ perception of the utility of sentiment analysis based on our findings from desktop research and focus group interviews, identifying the four categories (1) Utility for CPAs, (2) Impact on citizens, (3) Enhancement of two-way communication between citizens and CPAs and (4) Utility for disaster risk management. By validating items of the four categories, we assume to answer the research question of how a tool with AI-supported sentiment and entity extraction from crowdsourced data can help CPAs in leveraging their risk and disaster relief communication. Our research will hence make a significant contribution to citizens’ resilience and CPAs’ first response after earthquakes.