Graphology, the process of identifying the personality traits of an individual based on their handwriting, has been an established method for decades. However, traditional methods of handwriting analysis are highly subjective and time-consuming, often leading to inconsistent results. There is a large spectrum of industries in which this method can be effectively implemented such as employee profiling, crime investigation, education, and psychological assessment. Despite the potential applications, the manual analysis of handwriting samples remains inefficient, limiting its scalability and accuracy. A few years ago, this used to be a manual process wherein every handwriting sample would go through a manual verification, leading to a personality report that could be used to determine the behavioural trait. However, with the advancement of technology, we now have enough data to train our models, which can then be used to categorize handwriting samples into relevant classes with a higher accuracy. The research gap addressed by this study lies in the lack of automated, scalable, and accurate systems for handwriting-based personality trait prediction, specifically using advanced deep learning techniques. In the proposed system, we are initially inputting the handwriting sample images, which are first pre-processed. Techniques such as image resizing, gray scaling, and noise removal are used here for pre-processing. The model is then trained using multiple models such as Convolutional Neural Networks, Recurrent Convolutional Neural Network along with a couple of transfer learning implementations called Inception R-CNN and VGG16. To the best of our knowledge, this is one of the first attempts to combine these specific architectures for handwriting-based personality analysis. A comparative study of all the models is performed as part of the results. The samples are classified into personality traits: ‘Extraversion’, ‘Neuroticism’, ‘Openness’, ‘Agreeableness’, and ‘Conscientious’. We have also solved the data imbalance issue in our dataset with the help of data augmentation using Generative Adversarial Networks (GANs). Our novel approach achieves an 80% accuracy using the Recurrent CNN network, which outperforms traditional methods in both accuracy and scalability.