Model selection aims to find the best model in terms of accuracy, interpretability or simplicity. Here, we focus on evaluating model performance of Gaussian process models, which can be used for data analysis of uncertain data. While previous work uses methods like the Marginal log likelihood, AIC or nested sampling to perform model selection, they either lack performance or have significant runtime issues, limiting their applicability. We address these challenges by introducing multiple metrics based on the Laplace approximation, where we overcome an inconsistency occurring during its naive application which causes overfitting in terms of model selection. We show how AIC can fail to recognize the more appropriate model and perform large scale experiments to conclude that our Laplace-based metrics outperform the state of the art for fast model selection. Our model selection metrics allow fast and high quality model selection of Gaussian processes.