Scholarlink

In this article, Llama LLM (Large Language Model) is fine tuned with LORA (Low Rank Adaptation) to make it suitable for analyzing astronomical data. Instead of modifying the entire eight billion parameters of the Llama 3 model, we applied LoRA adapters to reduce training time and minimize original knowledge deterioration. The adapters operate as small information guides that can enter particular regions of the attention system to help it utilize language abilities for astronomical subject matters. The proposed scheme uses a compact 4-bit quantization scheme QLoRA (Quantized Low-Rank Adaptation) that condensed the model into a suitable format for quick training and running. Results indicate reduced training time to a few hours on high-end GPUs instead of traditional weeks or days. The fine tuned proposed model provides 85.2% Exact Match Score (EM) when compared with Llama 3 8B Chat HF v1 (Base Model) that reported 60.8 %. This research will facilitate readers robust guidelines towards modifying LLM for domain specific training data which allow fine-tuning while maintaining core functionality.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient LLM Framework Using LORA Based Fine Tuning to Analyze Textual Astronomical Data

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Sk Sahil Parvez,
  • Roushan Kumar,
  • MD Shaheer,
  • Snigdha Sen,
  • Pavan Chakraborty

摘要

In this article, Llama LLM (Large Language Model) is fine tuned with LORA (Low Rank Adaptation) to make it suitable for analyzing astronomical data. Instead of modifying the entire eight billion parameters of the Llama 3 model, we applied LoRA adapters to reduce training time and minimize original knowledge deterioration. The adapters operate as small information guides that can enter particular regions of the attention system to help it utilize language abilities for astronomical subject matters. The proposed scheme uses a compact 4-bit quantization scheme QLoRA (Quantized Low-Rank Adaptation) that condensed the model into a suitable format for quick training and running. Results indicate reduced training time to a few hours on high-end GPUs instead of traditional weeks or days. The fine tuned proposed model provides 85.2% Exact Match Score (EM) when compared with Llama 3 8B Chat HF v1 (Base Model) that reported 60.8 %. This research will facilitate readers robust guidelines towards modifying LLM for domain specific training data which allow fine-tuning while maintaining core functionality.