Supervised training of deep learning models based on a convolutional neural network architecture has been the preferred approach in available literature for brick segmentation in masonry buildings. Despite showing promising outcomes, the drawbacks stemming from expensive data acquisition and limited generalization capabilities pose significant challenges in the development and practical implementation of this solution. Instead, the present study addresses a pioneering approach that harnesses off the shelf artificial intelligence technology. This approach uses readily available foundation deep learning models that can be used for specific applications without the need of training with domain-specific data. A brick segmentation pipeline is proposed that couples three foundation models and apply it on various masonry façade images. Furthermore, the pipeline is extended with an automated approach for the quantification of building settlement profiles. This information on settlement is encapsulated in the brick segmentation masks and can serve as valuable input for damage analysis tools. The proposed brick segmentation pipeline exhibits outstanding performance in real-world conditions, by providing accurate segmentations in noisy images containing artifacts and different resolutions. It is also successfully applied to Google Street View imagery. The image-based quantification of building settlement is tested in a case study that involves two masonry terraced houses subjected to relatively large settlement. The resulting settlement profile is compared to bed-joint levelling measurements using a total station. A reasonably good agreement is observed, but improvements on the image calibration step (that establish the relation between pixel dimensions and real-world size and orientation) are needed. These promising results open the door for inexpensive, flexible, and scalable visual inspection techniques of masonry buildings.