Equipping personalities to dialogue agents can help to better engage end-users. However, how to profile personality remains an open research question due to the difficulties of obtaining real human data. As classic literary characters often encapsulate typical human personality traits, literature books has been used as a high quality data source to construct personality profiles for dialogue agents. Existing work mainly focuses on using external reviews and human experts’ annotations to profile character personalities. The in-text comments about the personality of characters in a literature book itself have been ignored. In this paper, we propose a new NLP task called character comments annotation to annotate the in-text comments about the personality of characters including dialogue utterances and surrounding text, paragraphs mentioning a character. We constructed new personality annotated dialogue datasets based on Gutenberg literature book project. We propose a workflow to automatically profile literary characters from literature novel books. Two personality profiling models have been proposed, including (i) psychological personality traits vocabulary-based spectrum (spectrums) approach and (ii) a tf-idf based words selection as a baseline approach. We applied the proposed personality models in dialogue response prediction tasks with ranking-based and generative dialogue agents. The results show that the fine-tuned dialogue agents with spectrums profiles surpass those trained without them by 2.5% (Hits 1@20) for ranking-based, and by 8% (Rouge-1) for generative agents. The implementation of the workflow with study-related resources is publicly available: https://github.com/nicolay-r/book-persona-retriever