In this study, we propose a novelty detection algorithm to enhance the performance of end-to-end few-shot document classification in real applications. The novelty detector eliminates ambiguous results and improves the precision of the final outcome. The detector can be integrated with various types of few-shot classifiers. The proposed method is evaluated on both private and public administrative datasets. Experimental results indicate that our proposed novelty detector efficiently removes ambiguous results of not only from few-shot categories but also from base categories, thereby significantly improving the precision across all categories. Furthermore, the best-performing few-shot classifier, when combined with our detector, reduces the error rate on the private dataset from 2.66% to only 0.96%, while maintaining a high recall of 95.64%. On the public RVL-CDIP dataset, the detector significantly lowers the error rate from 40%, which is impractical, to just 4.64%, making it feasible for real-world applications.