Local Differential Privacy (LDP) has emerged as a standard framework for privacy-preserving data collection. However, recent work [4] reveals that LDP protocols, e.g., Optimized Unary Encoding (OUE), etc., are vulnerable to data poisoning attacks, where malicious users can send carefully-crafted fake data to alter the estimated frequencies. To defend against such attacks, in this paper, we propose a novel scheme named Fine-filter, which serves as a plug-in module deployed on the collector side after data aggregation. In Fine-filter, users are divided into two groups by their reported data patterns. We believe that one group contains all the true users and the other includes both true and malicious users. By comparing the statistic information (e.g., frequency of each item) between two groups, we can locate the corrupt items and identify the malicious users with high confidence. Subsequently, the collector estimates item frequencies from the collected data after removing the data of (identified) malicious users. Experimental results demonstrate that Fine-filter can significantly mitigate the negative impact on estimation caused by poisoning attacks.