Visual localization is the problem of estimating the camera pose of a given query image within a known scene. Most state-of-the-art localization approaches follow a structure-based paradigm and use 2D-3D matches between pixels in a query image and 3D points in the scene for pose estimation. These approaches assume an accurate 3D model of the scene, which might not always be available, especially if only relatively few images are available to compute the scene representation. In contrast, structure-less methods only use 2D-2D matches and do not require any 3D scene model. However, they are also less accurate than structure-based methods. Although some prior works proposed to combine structure-based and structure-less pose estimation strategies, their practical relevance has not been shown. We analyze combining structure-based and structure-less strategies while exploring how to select between poses obtained from 2D-2D and 2D-3D matches, respectively. We show that combining both strategies improves localization performance in multiple practically relevant scenarios. In particular, the combined strategy allows to gracefully handle degradations in 3D scene model quality.