Scholarlink

The rise of Generative Artificial Intelligence (GenAI) has traditionally been dependent on high-performance GPUs, limiting accessibility for cost-conscious developers and researchers. This project explores an alternative approach by deploying Large Language Models (LLMs) on Intel AI laptops, focusing on optimizing inference and fine-tuning capabilities using Intel’s OpenVino toolkit. This study investigates the computational efficiency of CPU-based LLM execution, benchmarks performance across different model configurations, and implements optimization techniques such as quantization and parallel execution to enhance processing speed and resource utilization. The analysis highlights key performance trade-offs between accuracy, latency, and power consumption under varying optimization levels. Additionally, a streamlined pipeline for fine-tuning LLMs on Intel hardware is developed, ensuring practical adaptability for real-world applications. By providing a detailed performance analysis, an optimization framework, and comprehensive documentation, this research contributes to making GenAI more efficient and accessible through CPU-based inference, broadening its impact on consumer-grade hardware.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimizing Large Language Model Inference and Fine-Tuning on Intel AI Laptops Using OpenVino: A CPU-Centric Approach

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Afsar Khan,
  • Sailesh Rajagopalan

摘要

The rise of Generative Artificial Intelligence (GenAI) has traditionally been dependent on high-performance GPUs, limiting accessibility for cost-conscious developers and researchers. This project explores an alternative approach by deploying Large Language Models (LLMs) on Intel AI laptops, focusing on optimizing inference and fine-tuning capabilities using Intel’s OpenVino toolkit. This study investigates the computational efficiency of CPU-based LLM execution, benchmarks performance across different model configurations, and implements optimization techniques such as quantization and parallel execution to enhance processing speed and resource utilization. The analysis highlights key performance trade-offs between accuracy, latency, and power consumption under varying optimization levels. Additionally, a streamlined pipeline for fine-tuning LLMs on Intel hardware is developed, ensuring practical adaptability for real-world applications. By providing a detailed performance analysis, an optimization framework, and comprehensive documentation, this research contributes to making GenAI more efficient and accessible through CPU-based inference, broadening its impact on consumer-grade hardware.