In X-ray coronary angiography (XCA), accurate vessel extraction is critically important for the diagnosis of coronary artery disease. However, this task remains highly challenging due to the complexity of background structures and the presence of varying motion patterns with different intensities. In this work, a novel method is proposed for vessel layer extraction based on bilinear factor matrix norm robust principal component analysis and a Laplacian regularization is introduced to enhance the separation effect. For vessel layer images with uneven contrast distribution, we use a two-stage region growing (TSRG) method for vessel enhancement and segmentation. A region growing method is first applied to extract the main branches. Subsequently, an RLF filter is utilized to enhance and reconnect small fragmented segments. These two intermediate outputs are then combined to construct the final binary vessel mask. The proposed method has been validated on both clinical XCA image sequences and a publicly available third-party dataset. Qualitative and quantitative results demonstrate that the proposed approach outperforms several state-of-the-art methods in terms of both accuracy and robustness.