The CDT test for human brain health analysis is a very typical medical process for evaluation of human brain disease, it’s simple and easy to practice regardless of the condition of the context. It only needs patients or testing candidates to draw a required clock to indicate the time correctly, to see if the candidate can perform such a task within an appropriate time range. Scores will be given after the evaluation on the draws, from 0 to 5, in order to indicate the candidate’s brain can work well in terms of Alzheimer diagnosis. However, such a process can be determined by an assistant working with artificial intelligence, through deep learning. Those computer vision models such as VGG16 should be working with those drawn images to decide the score of the patients’ draw. Challenges come from many aspects, such as the quality of the acquired data, the adjustment of those models in terms of transferred learning, those hyperparameters settings, the multi-type class classifications difficulties due to the minor differences between each score gap. This paper discusses those problems and provides an initial study on using self-designed models, as well as well known models in transferred learning to identify those image scores. The data set is originally collected from the public USA government dataset.