Emotion cause extraction in conversations (ECEC) is an important task in emotion analysis, aiming to extract the text spans, i.e., parts in utterances, that reflect the causes of a certain type of emotion embedded in a target utterance in the conversation history. Since conversations are interactions between individuals, where one responds to the content from other participants, it is crucial to identify the potential responsive relations among utterances so as to build a response graph over the utterances to illustrate the underlying structure of the conversation and thus contribute to ECEC. However, existing studies, including the ones using large language models (LLMs), focus little on modeling the responsive relations when they perform the task. In this paper, we propose to improve ECEC with LLM and response graphing over utterances, where responsive relation decoding (RRD) process is proposed to identify responsive relations among utterances by learning from the responsive relations automatically extracted by off-the-shelf toolkits. Attentive relational graph convolutional networks (A-RGCN) are further applied to the response graph to weigh and leverage the utterances, where the encoded information is used as prompt to instruct the LLM to perform ECEC. Experimental results and analyses on English benchmark datasets demonstrate the effectiveness of the proposed approach, where our approach achieves state-of-the-art results on the datasets (Our code is available at https://github.com/synlp/ECEC-RG ).