Overfitting is a common problem in various artificial neural networks (ANNs) because of limited data, a complex model, or a lack of regularization. Chaos injection is one of the effective solutions to prevent overfitting for multilayer perceptron neural networks (MLPs). This paper presents an improved chaotic injection technique utilizing the enhanced multiparametric tent map (MTM). Using this heuristic technique, we choose a set of control parameters with the most uniformity according to the Kolmogorov-Smirnov (KS) method. When the MTM is used in training an ANN, the uniformity of each generated sequence is greatly reduced due to the random selection of the initial value of a sequence. To solve this problem, we generated all the sequences before training time using the MTM based on dataset size, number of neurons in each layer, batch size, and number of k-folds. Next, we normalized the sequences through a novel chaotic-sequence normalization and probability mass function (PMF) equalization procedure to ensure they were uniform, and finally, we stored them in a table. During training, the uniform sequences were read from the table and were injected into the learning fabric of the ANN. The experimental results demonstrated a superior accuracy of our proposed scheme in comparison with the latest techniques according to the following performance metrics: accuracy (ACC), F1-score (F1), negative-predictive value (NPV), positive-predictive value (PPV), sensitivity (SN), and specificity (SP).

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

High-Precision Method to Reduce Overfitting in ANNs Using Highly Uniform Chaotic Sequences

  • Samad Shirzadeh,
  • Ken Ferens,
  • Witold Kinsner

摘要

Overfitting is a common problem in various artificial neural networks (ANNs) because of limited data, a complex model, or a lack of regularization. Chaos injection is one of the effective solutions to prevent overfitting for multilayer perceptron neural networks (MLPs). This paper presents an improved chaotic injection technique utilizing the enhanced multiparametric tent map (MTM). Using this heuristic technique, we choose a set of control parameters with the most uniformity according to the Kolmogorov-Smirnov (KS) method. When the MTM is used in training an ANN, the uniformity of each generated sequence is greatly reduced due to the random selection of the initial value of a sequence. To solve this problem, we generated all the sequences before training time using the MTM based on dataset size, number of neurons in each layer, batch size, and number of k-folds. Next, we normalized the sequences through a novel chaotic-sequence normalization and probability mass function (PMF) equalization procedure to ensure they were uniform, and finally, we stored them in a table. During training, the uniform sequences were read from the table and were injected into the learning fabric of the ANN. The experimental results demonstrated a superior accuracy of our proposed scheme in comparison with the latest techniques according to the following performance metrics: accuracy (ACC), F1-score (F1), negative-predictive value (NPV), positive-predictive value (PPV), sensitivity (SN), and specificity (SP).