The rise of do-it-yourself (DIY) cosmetic procedures promoted on social media platforms such as TikTok has introduced new risks to public health, particularly as untrained individuals attempt at-home dermal filler injections. This study investigates the feasibility of detecting medical misinformation related to DIY fillers using machine learning techniques and examines the impact of annotation consistency on model performance. We collected and manually labeled 195 TikTok posts using a three-class schema: non-relevant, relevant-benign, and relevant-misinformation. Labels were assigned by both a medical expert and a technical contributor, with a subset re-labeled to assess intra-annotator agreement. Results showed moderate-to-substantial agreement within the same expert (κ = 0.624) but low agreement across annotators, revealing variability in label interpretation. A Random Forest classifier trained on different label subsets showed that annotations from the more internally consistent rater led to stronger model performance, particularly in precision. These findings underscore the importance of early investment in annotation quality and inter-rater validation when building AI systems for misinformation detection. We discuss the implications for public health surveillance and propose future work to scale content filtering and support qualitative review by experts.