Deep neural networks and supervised machine learning for medical image segmentation, including dermatology [10], require large pixel-wise annotated datasets for training, which can be difficult to obtain. Image classification, on the other hand, only requires a label for each image, which is often automatically provided with a medical diagnosis, but does not provide segmentation maps. However, in image-classification tasks, Explainable-AI (XAI) algorithms provide a means of identifying pixels in the original image that are part of the object or relevant structure. We propose to exploit this information for segmenting the images by building a network graph from XAI explanations and using the graph-cut algorithm for segmentation. Our approach is evaluated using the HAM10k [26] dataset, demonstrating its ability to segment skin lesions in dermatoscopic images without requiring pixel-annotated data for training. This makes our approach a cost-effective alternative in scenarios where annotated images are not available.