Portfolio optimization is the allocation of capital to different assets to meet predefined goals such as maximizing returns and minimizing volatility. Deep-Reinforcement Learning (RL) based solutions are often used for automated trading or for assisting human decision making; however, given the inscrutable nature of neural networks, interpreting an action taken by a deep-RL agent becomes nearly impossible, which poses a critical challenge in its widespread adoption in financial domain. In this paper, we propose a novel approach inspired by vision transformers, integrating a self-attention mechanism to generate saliency scores for feature relevance. Additionally, we introduce a method for training that optimizes both financial performance and interpretability by rewarding financial returns along with confident feature selection. We compare diverse algorithms such as Deep-Q Network (DQN), Reinforce, Actor-Critic, and Proximal Policy Optimization (PPO) for both continuous and discrete action spaces to empirically establish the interpretability and effectiveness of our approach. We have also compared our results with those obtained by using simple rule-based heuristics to strengthen our claims and finally conducted ablation study to quantitatively assess the effectiveness of our proposed saliency module. As a result, we observed an average performance improvement of 12.042% across the aforementioned RL algorithms when incorporating our saliency module, alongside enhanced interpretability achieved through the module’s feature selection capability. The data used for experimentation can be accessed at https://github.com/sharchit/interpretable-portfolio-optimization .