Multimodal Emotion Recognition in Conversations (ERC) is crucial for understanding human communication, aiming to infer speakers’ emotional states through verbal and nonverbal cues. While current graph-based models utilize Graph Neural Networks (GNNs) in conversation graphs, they often focus on developing complex neural network architectures and multimodal fusion processes, neglecting the optimization of the underlying graph structure. The graph structures where nodes are linked to a quantity of other nodes are of high complexity in computation but low efficiency in expressing temporal and local information for emotion inference. Inspired by strategic skills in reading comprehension, we propose a novel approach that transforms multimodal conversations into heterogeneous graphs with reduced edge density, which significantly decreases the computational cost of GNN operations. Accordingly, we complement this with our proposed Multimodal Graph Reasoning Network (MMGRN) to effectively process the optimized graph structure of rich temporal and local information. Our method outperforms the best graph model without recurrent neural network in two benchmark datasets, achieving 4.2% and 0.4% improvements in two typical multimodal ERC datasets MELD and IEMOCAP, respectively, while significantly increasing processing speed. This work demonstrates that improving the foundational graph structure, combined with efficient reasoning mechanisms, can lead to substantial advancements in multimodal emotion recognition tasks.