This paper proposes an Exponential Non-negative Matrix Factorization (ExpNMF) model for extracting discriminative non-negative features to achieve robust image representation. To solve the optimization problem, we develop a novel gradient descent algorithm with theoretically guaranteed convergence. Rigorous mathematical analysis derives a valid step size range, ensuring stable convergence behavior. Compared with most NMF-based algorithms, ExpNMF offers key advantages, including automatic step size adjustment along the negative gradient, inherent capability to handle mixed-sign data, and effective support for zero-value initialization. Experimental validation on diverse image datasets confirms consistent convergence within the theoretical step size bounds while demonstrating superior classification accuracy for both non-negative and mixed-sign data. These results demonstrate that Exp-NMF provides an effective feature extraction framework for real-world data applications.