Climate change impacts, social inequalities, and governance challenges increasingly impact municipal performance, creating challenges for financial institutions, insurers, and asset managers trying to evaluate sustainability-related risks. Unlike corporations, municipalities spread ESG-relevant information across various fragmented sources, including plans, bylaws, reports, and public outreach efforts. We present RAG4MuniESG, an AI-driven framework that combines Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) to facilitate ESG assessments specific to municipalities. Using a public-sector-oriented ESG scheme, the system semantically merges structured indicators with unstructured documents. We propose a hybrid AI-human approach wherein automated extraction and scoring are augmented with transparent and traceable justifications to support human-in-the-loop verification and auditability. RAG4MuniESG produces (1) aggregated ESG profiles for overarching risk screening and (2) detailed outputs with page-level references for auditing and oversight. Tested in three Canadian cities, it shows how such tool can convert fragmented municipal data into actionable ESG insights that meet the needs of financial and regulatory stakeholders.