Despite the advanced success of deep learning methods on tabular learning, the divergent researches on missing tabular data imputation and tabular data prediction will undoubtedly yield the suboptimal performance on downstream tasks involving incomplete tabular data. In this paper, we tackle the tasks of simultaneous tabular data prediction and imputation in a joint unified mask graph modeling paradigm, dubbed UMG4Table. We construct a skeleton graph along with auxiliary graphs to depict intricate relationships among entities, attributes, and labels. This approach results in a holistic tabular ensemble graph, which facilitates the generation of shareable representations, as well as unifying tabular data prediction and imputation with graph edge prediction. Additionally, we introduce unified mask graph learning to incorporate different types of graphs from a message passing perspective, which fully integrates information from them together and enhances the generalization ability of the model. Consequently, experiments in this paper cover both tabular data prediction and tabular data imputation in incomplete data. Our method not only advances the prediction performance in incomplete data by a significant margin, but also consistently performs on par with state-of-the-art imputation methods. Elaborate ablation studies and sensitivity analysis of parameters verify the effectiveness and robustness of our different strategies. Our code will be made publicly available.