<p>Currently, source-free domain adaptation has garnered significant attention and achieved promising research outcomes. It requires only a pre-trained source model and unlabeled target domain data to transfer knowledge and obtain a well-performing target model. Diversity and discrimination respectively measure the degree of distinction between different categories and the degree of aggregation among different samples of the same category, which have a crucial impact on the performance of the target model. Nevertheless, the current approaches in this aspect are relatively coarse. Hence, we propose a category-based method to enhance the diversity and discriminability of the target domain. Among them, we incorporate the source domain category relationship and the confidence of model prediction to optimize the calculation of category similarity and instance similarity. Furthermore, in object recognition, there are certain categories that are prone to confusion, and samples belonging to these categories are highly susceptible to erroneous classification. To solve this problem, we propose a novel approach, encompassing the screening of hard categories and the optimization of hard samples associated with these categories. Extensive experiments on three benchmark image datasets demonstrate the effectiveness of our approach.</p>