Scholarlink

This paper conducts a comparative study of large language models, namely Llama3-70B-8192, Llama3-8B-8192, Gemma2-9B-it, and Mixtral-8 × 7B-32768 on structured data processing within the Retrieval-Augmented Generation framework. Given the mounting need for in-depth, accurate, and efficient analysis of structured data in finance and healthcare applications, we benchmark these models concerning retrieval accuracy, logical reasoning, and summarization tasks. As for the key findings, model-specific strengths are evident in this study. Gemma2-9B-it has the best result in terms of precision in exact-match retrieval with a Cosine Similarity score of 0.6906, Mixtral-8 × 7B-32768 tops the lists of logical reasoning with a semantic similarity score of 0.4651, and summarization, where Gemma2-9B-it tops the results, being a leader in content retention via a ROUGE-1 score of 0.1175, and Llama3-70B-8192 maintained the coherence of the phrases via a ROUGE-2 score of 0.0304. These results guide the next applications, with the taste of using models to their optimum fit in achieving accuracy and efficiency through enhancing structured data contexts.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Benchmarking Large Language Models for Structured Data Processing

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Lakshay Sharma,
  • Soumil Suri,
  • Sankalp Mathur,
  • Bhawna Rawat

摘要

This paper conducts a comparative study of large language models, namely Llama3-70B-8192, Llama3-8B-8192, Gemma2-9B-it, and Mixtral-8 × 7B-32768 on structured data processing within the Retrieval-Augmented Generation framework. Given the mounting need for in-depth, accurate, and efficient analysis of structured data in finance and healthcare applications, we benchmark these models concerning retrieval accuracy, logical reasoning, and summarization tasks. As for the key findings, model-specific strengths are evident in this study. Gemma2-9B-it has the best result in terms of precision in exact-match retrieval with a Cosine Similarity score of 0.6906, Mixtral-8 × 7B-32768 tops the lists of logical reasoning with a semantic similarity score of 0.4651, and summarization, where Gemma2-9B-it tops the results, being a leader in content retention via a ROUGE-1 score of 0.1175, and Llama3-70B-8192 maintained the coherence of the phrases via a ROUGE-2 score of 0.0304. These results guide the next applications, with the taste of using models to their optimum fit in achieving accuracy and efficiency through enhancing structured data contexts.