Comparison of LLMs on Financial Professional Tests
摘要
Large language models (LLMs) are being used in an increasing number of areas. One such industry is finance. The practical application of models, including the financial sector, is inextricably linked to their evaluation. Currently, there are a large number of benchmarks that test the performance of models in the financial field. However, there are also professional exams in this field that can be used to test LLMs. Earlier studies looked at a small number of exams, mainly the CFA (Chartered Financial Analyst). This paper conducts a comparative analysis of the performance of LLMs on an expanded number of professional exams in the field of finance. Five exams were considered, of which three were in English and two were in Russian. A total of 2954 test questions were collected. Six LLMs were evaluated: GigaChat 2 Max, DeepSeek-R1, DeepSeek-V3, Llama 4 Maverick, GPT-4o-mini, and Qwen3 235B (reasoning and non-reasoning modes). The best models were DeepSeek-R1 and Llama 4 Maverick, which showed the highest results in 3 and 2 exams, respectively. Approximate threshold levels were passed in all exams except for the Russian-language FFMS (Federal Financial Markets Service) exam. This may indicate a fairly high level of LLMs knowledge in the field of finance, which will help expand the practical application of LLMs.