The legal industry is characterized by the presence of large volumes and complex documents. Given the continuous evolution of these documents, there is a growing interest in automating the processing of legal texts to streamline compliance. One key step of this process is the extraction of legal entities. State-of-the-art methods for legal entity extraction, including rule-based systems, Bi-LSTM, and BERT, require substantial annotated data to be effective, a task that is time-intensive for domain experts. With the rise of Large Language Models (LLMs), research has increasingly focused on leveraging their capabilities and exploring zero-shot approaches. In this paper, we present a hybrid system that distils GPT-4 knowledge through rule-based methods into a CamemBERT model. This approach not only reduces the need for expert involvement compared to the standard CamemBERT system but also outperforms the GPT-4-only system, enhancing the F1 score for legal entities by 9–24% points.