The inherent cognitive capability enables humans to navigate complex decision spaces effectively. Evidence reveals that the concurrent reasoning of the prefrontal cortex is vital for decision-making in the human brain. Inspired by this brain mechanism, we propose a brain-inspired deep reinforcement learning approach, called Parallel Inference Learning (PIL). Specifically, the proposed method infers the exploitation of existing strategies and the exploration of new possibilities in parallel to balance exploration and exploitation. Therefore, the agent selects an appropriate strategy to guide online decision-making, which can optimize the online learning process. To illustrate the optimization performance, we conduct a case study focusing on motion planning tasks in high-dimensional continuous spaces. The results show that PIL outperforms the baselines in terms of three representative metrics, which confirms the potential of emulating human-like capabilities to enhance the efficiency of decision-making.