Scholarlink

The last couple of years were marked by the rise of LLMs (Large Language Models) that are shown to bring important developments in various sectors such as technology, education or customer service. Generative AI also impacts other areas such as content generation, data analysis, and can be used to automate various systems. From a security point of view, initial applications of LLMs improve customer support services - be it direct customer support or security operation centers - and also the way an IT administrator connects with SIEM (Security Information and Event Management) or XDR (Extended Detection and Response) products. However, there were fewer practical attempts to use LLMs to improve the detection rate or a security product at large. The current paper explores this direction by analyzing the detection rate of several LLMs (ChatGPT, LLama, Mistral, Orca2, Falcon) against a recent set of macro VBA (Visual Basic for Application) malware. The respective performance levels are discussed. Other relevant issues, such as inference cost, memory and space requirements and practicality from a security product point of view, are also considered. Even if the models used in our research do not reach adequate results in terms of detection rate they can still be used in security operation centers to assist a security researcher to analyse malicious content. Furthermore, additional fine-tuning might improve the accuracy of such models to a point where they may become a good choice for various security tasks.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On the Properness of Large Language Models for Malware Detection

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Lupaşcu Marilena,
  • Viţel Silviu Constantin,
  • Gavriluţ Dragoş Teodor,
  • Luchian Henri

摘要

The last couple of years were marked by the rise of LLMs (Large Language Models) that are shown to bring important developments in various sectors such as technology, education or customer service. Generative AI also impacts other areas such as content generation, data analysis, and can be used to automate various systems. From a security point of view, initial applications of LLMs improve customer support services - be it direct customer support or security operation centers - and also the way an IT administrator connects with SIEM (Security Information and Event Management) or XDR (Extended Detection and Response) products. However, there were fewer practical attempts to use LLMs to improve the detection rate or a security product at large. The current paper explores this direction by analyzing the detection rate of several LLMs (ChatGPT, LLama, Mistral, Orca2, Falcon) against a recent set of macro VBA (Visual Basic for Application) malware. The respective performance levels are discussed. Other relevant issues, such as inference cost, memory and space requirements and practicality from a security product point of view, are also considered. Even if the models used in our research do not reach adequate results in terms of detection rate they can still be used in security operation centers to assist a security researcher to analyse malicious content. Furthermore, additional fine-tuning might improve the accuracy of such models to a point where they may become a good choice for various security tasks.