Background <p>Melanoma is a highly invasive and metastatic skin cancer lacking effective options for early diagnosis and treatment. To address these challenges, this study aimed to synthesize a novel near-infrared fluorescent probe and research the antitumor mechanism of melanoma.</p> Methods <p>We have designed and synthesized a novel near-infrared fluorescent probe, IR-546. This study explores imaging capabilities, anti-tumor effects, and underlying mechanisms of IR-546 in melanoma xenograft models.</p> Results <p>In vitro, IR-546 selectively accumulates in melanoma cells, demonstrating robust tumor imaging and potent cytotoxicity by targeting mitochondria, generating reactive oxygen species (ROS), and disrupting mitochondrial membrane potential, activating the mitochondrial apoptotic pathway. Additionally, IR-546 exhibits anti-metastatic properties in vitro. In vivo, using A375 cell line-derived melanoma xenograft models, IR-546 showed significant anti-tumor effect and biosafety. Western blot analyses of both in vivo and in vitro revealed that IR-546 induces apoptosis and inhibits metastasis of melanoma by activating the mitochondrial apoptosis pathway, suppressing the AKT/GSK3β signaling pathway, and downregulating the β-catenin signaling pathway and its downstream targets. Immunohistochemistry and immunofluorescence further confirmed that IR-546 suppressed the expression of key proteins in the AKT/GSK3β pathway in vivo.</p> Conclusions <p>Collectively, these findings highlight IR-546 as a promising tool for both imaging and treatment of melanoma, with the potential to induce apoptosis and inhibit metastasis of melanoma through modulation of the AKT/GSK3β pathway.</p> Graphical Abstract <p>Summary of the diagnostic and anti-tumor effects of IR-546 in melanoma.</p> <p></p>