What Makes a 7 a 7? Exploration into a Neural Network’s Features of Interest
摘要
Neural networks (NNs) are increasingly used in critical domains such as medical diagnosis, autonomous driving, and financial applications. However, their complexity limits transparency and interpretability. The lack of explainability poses a risk for trust in AI systems. This research contributes to Explainable Artificial Intelligence (XAI) by exploring visualization techniques to help users understand why NNs make certain decisions. Specifically, we visualize node activations and edge weights within fully connected feedforward NNs trained on the MNIST dataset. These visualizations show the flow of data through the network. Our study stems from the following question: “Why does a neural network classify a given input as a 7?” Our findings suggest that NNs rely on partial feature recognition, which provides insight into their internal reasoning. To obtain a desired output, we simply activate the right nodes that cause the right flow of information through the network. Thus, visualization serves as an intuitive and powerful tool for NN interpretation, contributing to greater transparency in AI systems.