Instruct-Code-Llama: Improving Capabilities of Language Model in Competition Level Code Generation by Online Judge Feedback

General large language models (LLMs), represented by ChatGPT, have demonstrated an impressive ability in software development tasks such as code completion, comment generation, and unit test generation. However, when evaluated on competition level programming problems, these models still underperform. We observed that conventional models prefer to convert problem descriptions directly into codes, which results in the inability to handle some edge cases. We think this problem occurs because the training data contains incorrect code that only passes some of the test cases, but not all. To improve the capabilities of language models in competition level code generation, we introduced a novel fine-tuning approach called Reinforcement Learning with Online Judging Feedback (RLOJF) to align the generated code with the correct code. Then we collect a dataset including online judge feedback from Codefroces and base on this dataset train a new model called Instruct-Code-LLaMA, under the same conditions, the generated code exhibits a 0.29% increase in average accuracy on the APPs dataset.