Patient-specific cardiovascular simulations have become an integral part of cardiovascular research. A primary factor hindering large patient cohort studies and clinical impact of cardiovascular simulations is their dependence on accurate patient-specific three dimensional geometric models which remain time-consuming and costly to construct from medical image data. Methods have been proposed to automate the model construction process, for either vascular or cardiac purposes. We propose a novel method, MeshGrow, that, automatically, reconstructs both the cardiac chambers as well as the aorta and its main sub-branches, and returns a simulation ready mesh with defined aortic valve connecting the two. We deploy two different methods of model construction for the cardiac chambers and vascular regions independently to address the specific challenges involved with each. The method is a two step approach; 1) meshing the cardiac structures and 2) growing the vasculature out from it. We present test results of our method on three CT image data and compare to ground truth manually constructed models. Additionally, we compare results with state-of-the-art methods. Results show that MeshGrow achieves higher metric scores than benchmark methods on test set. With this work, we demonstrate the advantage of anatomy specific modeling approaches for patient-specific cardiovascular simulation.