GANet - Learning Tabular Data Using Global Attention

Deep Learning has proved its efficiency in various data types (image, video, text, audio, etc.). Various Deep Learning methodologies also have made significant strides, exhibiting better performance levels in the tabular data. Still, their practical utilization in real-world scenarios remains limited because their accuracy and interpretability are not comparable to classical methods, although the application of Deep Learning is attractive - building a multi-modal and end-to-end pipeline, utilizing transfer learning or applying self-supervised learning. In this work, we introduce GANet (Global Attention Neural Network) learning the target function on tabular data by utilizing global attention layers: we modify the self-attention mechanism to capture the shared interaction between features for the dataset. Through empirical results, we demonstrate that our novel model, GANet, consistently archives state-of-the-art performance and achieves better interpretability compared to prior Deep Learning models.