The recognition of oracle characters is meaningful for exploring ancient Chinese culture. Due to the poor quality of scanned oracle characters, recognition is a difficult task. Although many works try to improve the recognition performance, they overlook the important fact that oracle characters are hieroglyphic. In this paper, we utilize this fact by ProtoPNet to improve the capability of recognizing oracle characters. In particular, we propose OracleProtoPNet, which mimics the human expert’s recognition of oracle characters. Our OracleProtoPNet recognizes the oracle characters by comparing the similarity between input images and learned prototypes. To further improve recognition accuracy, we combine our model with two modules. Specifically, we introduce a multi-scale channel attention module to reduce noise interference. Additionally, we aggregate the activation value scores to avoid classification interference. Even if one patch has high similarity to a prototype of a certain class, the differences in other patches will separate them into different categories. To compare multiple prototypical networks and unsupervised domain adaptation networks, we conduct extensive experiments on the Oracle-241 dataset. The experimental results demonstrate that our network achieves SOTA accuracy on the scanned data while maintaining interpretability.