Social media platforms are crucial for modern communication, but they are vulnerable to the dissemination of harmful content that can negatively impact users and encourage negative interactions. To tackle this problem, a new approach was created by integrating Large Language Models (LLMs) with Reinforcement Learning (RL) to enable instant identification of toxic content and flexible moderation. Utilizing a pre-trained language model for detecting nuanced toxicity, the system incorporated an RL agent that adaptively modified detection thresholds according to user feedback. The model was able to improve its predictions by learning from both right and wrong classifications through the reinforcement mechanism. The findings showed a rise in accuracy in classification and a decrease in moderation errors, proving this method as a feasible, expandable option for improving online content moderation systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time Toxicity Detection and Mitigation in Social Media Using Reinforcement Learning

  • Prabal Manhas,
  • Joydeep Chandra,
  • Ramanjot Kaur,
  • Rashi Sahay

摘要

Social media platforms are crucial for modern communication, but they are vulnerable to the dissemination of harmful content that can negatively impact users and encourage negative interactions. To tackle this problem, a new approach was created by integrating Large Language Models (LLMs) with Reinforcement Learning (RL) to enable instant identification of toxic content and flexible moderation. Utilizing a pre-trained language model for detecting nuanced toxicity, the system incorporated an RL agent that adaptively modified detection thresholds according to user feedback. The model was able to improve its predictions by learning from both right and wrong classifications through the reinforcement mechanism. The findings showed a rise in accuracy in classification and a decrease in moderation errors, proving this method as a feasible, expandable option for improving online content moderation systems.