Implementation of new normalisation-based feature transformation methodology for multi-task learning using residual TCN with multi-channel and spatial attention
摘要
Multi-task learning (MTL) has reached significant attention because of its capability to allow the sharing of the knowledge between related tasks to enhance overall performance. By leveraging shared representations, MTL enables the development of more robust models for each task. However, implementing MTL in real-world scenarios poses several challenges. One critical issue is the imbalance of task contributions during training. When signals from multiple tasks are integrated into a shared network, tasks with larger gradient magnitudes tend to dominate the learning process. This leads to biased learning, where tasks with smaller gradients are underrepresented, adversely affecting the method’s simplification capability. To deal with this problem, a novel feature transformation methodology is proposed to support MTL using specifically curate datasets. The process begins with data acquisition from the MNIST dataset, followed by a data cleaning phase to eliminate unwanted outliers and inconsistencies. After cleaning, the feature transformation methodology is applied. This involves an initial normalization step to standardize the data. Following this, Optimally Weighted Principal Component Analysis (OWPCA) is employed for variable selection. Unlike traditional Principal Component Analysis (PCA), OWPCA assigns weight optimization to the original variables depends on their applicable material to the analysis, thereby enhancing dimensionality reduction and noise filtering while preserving important data characteristics. The optimal weighting in OWPCA is achieved using the Revised Mutation Rate-based Drawer Algorithm (RMR–DA). Inspired by the original Drawer Algorithm (DA), which simulates the procedure of choosing items from the multiple receptacles to find an optimal combination, RMR–DA introduces an adaptive mutation rate to balance exploration and exploitation more effectively. This enhancement significantly improves the search process, particularly in solving complex, real-world optimization problems. The refined feature set generated through this process is then input into the multi-task classification framework. For classification, a Residual Temporal Convolutional Network with Multichannel and Spatial Attention (RTCN–MSA) is employed. This model integrates the capabilities of Temporal Convolutional Networks (TCNs), Residual Networks, and Attention Mechanisms. Temporal Convolutional Networks (TCNs) are efficient in obtaining long-term dependencies in sequential information with parallel processing and stable gradients. Residual connections address the vanishing gradient problem, allowing the construction of deeper, more powerful models. Meanwhile, the inclusion of multichannel processing and spatial attention enables the network to adaptively aims on the most important components of the input data, further enhancing performance. This comprehensive MTL framework is designed to handle large datasets and perform complex classification tasks efficiently. Finally, the performance analyses are carried out on the developed model and compared with existing approaches to evaluate its effectiveness. The wide scope accuracy of the developed RTCN-MSA method is 89.57% in the MTL strategy using the Fashion-MNIST Dataset, which is better over other traditional methods. Thus, it proved that the integration of normalisation-based feature transformation with the RTCN-MSA architecture significantly enhances the accuracy, generalization, and robustness capability of an MTL across multiple tasks. Through addressing an imbalance in task contributions through effective feature weighting and attention-driven learning, the proposed methodology leads to precise and reliable classification outcomes. This demonstrates the model's suitability for real-world, multi-label classification problems involving large and complex datasets. This method effectively addresses the challenges of learning diverse tasks from complex data. By employing this, the model can accurately determine both temporal and spatial connection within the data, while simultaneously enhancing its ability to learn from various task-specific information. The integration of normalization further assists to the method’s generalization and stability capacities. The RTCN-MSA is more precisely accurate in implementing accurate classification outcomes and good at minimizing faults, specifying the model’s performance and implementing a direction for enhancement than other existing methods.