The spread of misinformation via fake news significantly affects public perception and threatens the democratic framework. Addressing this challenge requires advanced computational techniques capable of understanding and processing the nuances of human language. This study refines the BERT architecture to enhance misinformation detection by integrating non-linear layers and optimization strategies, improving accuracy, precision, recall, and F1-score. Novelty: This work advances the BERT model with these key contributions: (A) Non-linear Layers: Adds non-linear layers to capture complex data patterns, enhancing the model beyond traditional BERT. (B) Implementation of GELU Activation Functions: Utilizes Gaussian Error Linear Unit (GELU) activation functions for their superior ability to handle complex, non-linear relationships, which is uncommon in standard BERT-based models. (C) Advanced Optimization Techniques: Uses dropout and hyperparameter tuning to reduce overfitting, enhance generalization, and optimize accuracy, precision, recall, and F1-score. Data Preparation: A balanced dataset of labeled news articles (FAKE or TRUE) was collected from credible sources. The dataset underwent preprocessing steps, including cleaning, normalization, and transformation into numerical format using BERT embeddings. Model Development: This research builds upon earlier studies where Hugging Face Transformers were tailored for fake news detection (Misra et al. in 2023 International conference on data science, agents and artificial intelligence (ICDSAAI), Chennai, India, pp 1–5, 2023) and benchmarked against models such as SVM and passive aggressive classifiers (Misra et al. in Data management, analytics and innovation. ICDMAI 2024. Lecture Notes in Networks and Systems, Springer, Singapore, 2024). This study uses the pre-trained BERT-base-uncased model, known for its advanced natural language understanding. Key enhancements included: (1) Dropout Layers: Implemented before and after non-linear activation functions to mitigate overfitting. (2) Intermediate Linear Layer: Added to transform intermediate representations. (3) Gaussian Error Linear Unit (GELU) Activation: Used for non-linear processing to capture complex patterns. (4) Output Classifier: A final linear layer classifies news as ‘FAKE’ or ‘TRUE’. Optimization: The model was fine-tuned on a labeled dataset, optimizing the learning rate and hyperparameters for better convergence. Training and Evaluation: The data was split into training and testing sets, with performance evaluated using accuracy, precision, recall, F1-score, and ROC-AUC. In May 2024, in-house tests on NVIDIA computes via Azure ML showed significant detection performance improvements. The proposed model achieved the following: • Accuracy: 98.18% • F1 Score: 0.9814 • Recall: 0.9775 • Precision: 0.9854 • Loss: 0.1103 • Runtime and Efficiency: Reasonable runtime and efficiency metrics. These results indicate that the enhanced nonlinear BERT model effectively identifies fake news with high reliability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficiency-Optimized Non-linear Transformers for Fake News Detection: A Case Study

  • Shamik Misra,
  • Purna Chandra Panda

摘要

The spread of misinformation via fake news significantly affects public perception and threatens the democratic framework. Addressing this challenge requires advanced computational techniques capable of understanding and processing the nuances of human language. This study refines the BERT architecture to enhance misinformation detection by integrating non-linear layers and optimization strategies, improving accuracy, precision, recall, and F1-score. Novelty: This work advances the BERT model with these key contributions: (A) Non-linear Layers: Adds non-linear layers to capture complex data patterns, enhancing the model beyond traditional BERT. (B) Implementation of GELU Activation Functions: Utilizes Gaussian Error Linear Unit (GELU) activation functions for their superior ability to handle complex, non-linear relationships, which is uncommon in standard BERT-based models. (C) Advanced Optimization Techniques: Uses dropout and hyperparameter tuning to reduce overfitting, enhance generalization, and optimize accuracy, precision, recall, and F1-score. Data Preparation: A balanced dataset of labeled news articles (FAKE or TRUE) was collected from credible sources. The dataset underwent preprocessing steps, including cleaning, normalization, and transformation into numerical format using BERT embeddings. Model Development: This research builds upon earlier studies where Hugging Face Transformers were tailored for fake news detection (Misra et al. in 2023 International conference on data science, agents and artificial intelligence (ICDSAAI), Chennai, India, pp 1–5, 2023) and benchmarked against models such as SVM and passive aggressive classifiers (Misra et al. in Data management, analytics and innovation. ICDMAI 2024. Lecture Notes in Networks and Systems, Springer, Singapore, 2024). This study uses the pre-trained BERT-base-uncased model, known for its advanced natural language understanding. Key enhancements included: (1) Dropout Layers: Implemented before and after non-linear activation functions to mitigate overfitting. (2) Intermediate Linear Layer: Added to transform intermediate representations. (3) Gaussian Error Linear Unit (GELU) Activation: Used for non-linear processing to capture complex patterns. (4) Output Classifier: A final linear layer classifies news as ‘FAKE’ or ‘TRUE’. Optimization: The model was fine-tuned on a labeled dataset, optimizing the learning rate and hyperparameters for better convergence. Training and Evaluation: The data was split into training and testing sets, with performance evaluated using accuracy, precision, recall, F1-score, and ROC-AUC. In May 2024, in-house tests on NVIDIA computes via Azure ML showed significant detection performance improvements. The proposed model achieved the following: • Accuracy: 98.18% • F1 Score: 0.9814 • Recall: 0.9775 • Precision: 0.9854 • Loss: 0.1103 • Runtime and Efficiency: Reasonable runtime and efficiency metrics. These results indicate that the enhanced nonlinear BERT model effectively identifies fake news with high reliability.