Magnetic resonance imaging (MRI) enhanced by the gado-linium-based contrast agents (GBCAs) is crucial in the assessment and management of cancer. However, the use of GBCAs introduces additional costs and raises potential safety concerns, including the risk of gadolinium accumulation in brain. Several generative learning methods based on GANs and diffusion models have been proposed to generate contrast-enhanced MRI from non-contrast-enhanced MRI. However, GANs face challenges such as gradient vanishing and mode collapse. Diffusion models also face several challenges, such as generation instability and long sampling times. In this paper, we propose a controllable flow matching (CFM) model for efficient synthesis of 3D contrast-enhanced brain MRI with fine-grained details of targets of interests. CFM adopts a straight-line generation path, enabling the generation of images in a single step. We design a multi-stage training strategy integrating controllable constraints to ensure that such a single-step sampling generating contrast-enhanced MRI meet specific controllable conditions. Our CFM model has been evaluated on both the BraTS2023 and an in-house datasets. Experimental results demonstrate that CFM led to state-of-the-art image generation and tumor delineation performance with promising generalizability. Our codes can be found at https://github.com/ladderlab-xjtu/CFM .