SAMACO_FS: feature selection for high-dimensional few instances using ant colony optimization algorithm and self-attention mechanism
摘要
Feature selection (FS) remains a critical challenge in data mining for high-dimensional data with few instances, especially in the field of biomedical. Although ant colony optimization algorithm(ACO) has achieved good performance in this topic, which faces tough challenges in the setting of heuristic factor (HF), updating mechanisms of pheromone and computational scalability for ultra-high-dimensional data. To alleviate these issues, this paper proposes an improved ACO algorithm based on the self-attention mechanism (SAMACO), where the feature importance weight scores calculated by the self-attention mechanism are used as the HF for ACO. The improved probabilistic selection mechanism incorporates the idea of the roulette wheel selection, optimizing the selection process according to feature dimensions to control the number of feature subsets. The dynamic updating mechanism of pheromone integrates the elite strategy, as the number of iterations increases, the intensity of updating pheromone for ants with fitness in the top 10% also increases. Finally, the superiority of SAMACO is verified on 13 high-dimensional datasets with few instances in the biomedical field. Compared with the average performance of 9 representative algorithms such as GOOSE, NPDOA and ACO-RNN, SAMACO maintains the advantage in accuracy while reducing the feature dimension by 90.6% and improving computational efficiency by 75.3%, which indicates that SAMACO provides a potential solution to FS for high-dimensional data with few instances.