In this paper, we propose a method to classify displaced object images using Chat GPT. First, GPT-4o generates a caption for an image. However, there are images with no generated captions. These are failed images, such as ones that mistakenly detect the edges of a desk, hands, or light reflections as displaced objects. These were put into an “unclear cluster” to remove them from further processing. The generated captions were converted into vectors using SentenceTransformers, a Python framework for text embeddings. The converted vectors were classified into classes (our proposed method) using the K-means method. The clustering results using the proposed method were compared with that of vector clustering generated from images using a deep learning model (conventional method). The proposed method was shown to have high accuracy when identifying both failed images and when clustering images. Furthermore, while participants classified two or three different replacement object images into the same box, the proposed method also created folders for the same object combinations. Usually, objects with different appearances are recognized as different objects in image feature classifying; however, in the captioned features classification process, the images of objects with the same purpose were classified into the same cluster, even if the images were unclear or had differing appearances. We suggest that classification via caption vectors is based on higher-order concepts than the classification of image features.