Visual Affordance Recognition: A Study on Explainability and Interpretability for Human Robot Interaction

Deep learning-based affordance research is motivated by the need to achieve higher degrees of automation and accuracy. On the contrary, the concept of affordance originated to explain the intuitive intelligent responses of humans in their interaction with the environment. With a state-of-the-art supervised deep learning model, achieving high accuracy in an object affordance recognition challenge is excellent and normal nowadays. Does it ensure that this model understands object affordance correctly and explains its decisions to users? Deep learning algorithms and architectures for affordance research must possess an understanding of their own. We present a brief study on the existing literature on explainable affordance research and offer few suggestions to improve the explainability and interpretability of current deep learning methods for affordance detection. Three pretrained vision models are considered for supervised object affordance classification without having affordance heatmaps as teaching signal. The output of these models obtained after the experimentation over modified CAD-120 dataset is fed to smooth grad-cam \(++\) for post hoc explainability analysis. These experiments lead to a proposal of a framework for object affordance classification.