This paper addresses the computational challenges inherent in image feature extraction, a process that is computationally expensive due to the high dimensionality and large volume of data. Traditional stand-alone or GPU-based machines often face limitations in processing capacity, memory constraints, and scalability issues when dealing with batch feature extraction tasks. These challenges necessitate the exploration of distributed computing platforms for enhanced efficiency and scalability. We propose a novel approach to image feature extraction utilizing PySpark, a distributed computing system, to harness the power of cluster computing. Focusing on the Local Binary Pattern (LBP) technique, known for its effectiveness in texture classification, we adapt this algorithm to operate within a distributed environment. By leveraging PySpark’s ability to parallelize tasks across multiple nodes, we significantly reduce computation time and resource constraints, enabling the processing of large-scale image datasets more efficiently. Our experiments, conducted on the LFW image datasets, demonstrate a substantial improvement in speed, achieving an N-fold increase in processing speed as the number of nodes is scaled from 1 to N. This scalability showcases the potential of distributed computing platforms like PySpark in overcoming the computational hurdles of image feature extraction, offering a scalable and efficient solution for large-scale image analysis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Computational Optimization of Image Feature Extraction Algorithms on PySpark and Hadoop

  • Arshi Jamal,
  • K. Ramesh

摘要

This paper addresses the computational challenges inherent in image feature extraction, a process that is computationally expensive due to the high dimensionality and large volume of data. Traditional stand-alone or GPU-based machines often face limitations in processing capacity, memory constraints, and scalability issues when dealing with batch feature extraction tasks. These challenges necessitate the exploration of distributed computing platforms for enhanced efficiency and scalability. We propose a novel approach to image feature extraction utilizing PySpark, a distributed computing system, to harness the power of cluster computing. Focusing on the Local Binary Pattern (LBP) technique, known for its effectiveness in texture classification, we adapt this algorithm to operate within a distributed environment. By leveraging PySpark’s ability to parallelize tasks across multiple nodes, we significantly reduce computation time and resource constraints, enabling the processing of large-scale image datasets more efficiently. Our experiments, conducted on the LFW image datasets, demonstrate a substantial improvement in speed, achieving an N-fold increase in processing speed as the number of nodes is scaled from 1 to N. This scalability showcases the potential of distributed computing platforms like PySpark in overcoming the computational hurdles of image feature extraction, offering a scalable and efficient solution for large-scale image analysis.