The main aim of classification algorithms which can be binary or multi-dimension is to select “worthy” data for annotation, to achieve better performance with less labeled trained data. The standard CNN and fast CNN algorithms require large datasets to extract features and to build the convolution layer. Usually, models are built by choosing the “worthy” data by the deep reinforcement learning algorithm, which can be used to provide better classification and to extract features from images. To train a model, which have the “worthy” classification algorithm, an approach called deep Q-learning is used. The state-action principle will provide the decision for the classification algorithm where, the image features will serve as “state” and the output of Q-network to decide the “action”. This paper enhances the classification ability by means of reducing the time complexity. Highest classification accuracy is achieved by implementing Q-Learning with CNN, which uses a smaller number of labeled datasets and perform large classifications.