GA-DE: an integrated meta-heuristic approach for optimizing feedforward neural networks

Feedforward neural networks (FNNs) are among the most widely used artificial neural network models, which have consistently demonstrated excellent performance in various fields, such as classification and regression tasks. However, the performance of FNNs does mainly depend on their structure (i.e., the number of hidden layers and that of neurons for each hidden layer) and initial weights, and they are usually required to manually determine via extensive tuning experiments in advance, thereby seriously limiting their practical applicability. To address this issue, this paper proposes an integrated meta-heuristic intelligent algorithm using genetic algorithm (GA) and differential evolution (DE). To satisfy the dynamic adjustment of network structure during the search process, a novel mixed encoding strategy is devised to represent both the structure and initial parameters of network model. Moreover, a combined offspring generation strategy is designed to explore more promising networks using GA and DE. Specifically, GA is utilized to search the optimal network structure, while DE is employed to refine its associated parameters. Following them, the potential information of different network models can be effectively interacted and shared, thus improving the search effectiveness and efficiency of algorithm. Finally, the performance of the proposed algorithm is validated by comparing with eight existing algorithms on twelve commonly used classification datasets and a practical problem, named air quality index prediction. Experimental results demonstrate that the proposed algorithm exhibits superior performance.