A problem of determining real efficiency of the Tall Array method in computing statistics of numeric data is considered. Random arrays of specified numeric types are generated, for which computational time is measured by applying ordinary and Tall Array MATLAB operators in computing statistics of the generated arrays. The main finding is, for two parallel processor workers, the Tall Array method becomes efficient if an array of no fewer than a 10 to 35 million entries is to be processed. The Tall Array minimum and maximum are similarly efficient in processing arrays with double and single precision, both real-valued and complex-valued, and their efficiency is comparable to that of the Tall Array standard deviation. The efficiency threshold of the Tall Array covariance is 1.75 to 5 times less than that of the Tall Array standard deviation. The Tall Array mean is the least efficient for non-integer representation. The Tall Array operators of minimum, maximum, and mean are similarly efficient on unsigned and signed integers. Nevertheless, the Tall Array mean operator appears to be more efficient on signed integers. Moreover, as the number of bits is increased, its efficiency grows, where the efficiency threshold decreases from 20 million entries for int8 down to 5 million entries for int64.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Tall Array Method Efficiency in Computing Statistics of Big Data

  • Vadim Romanuke,
  • Patrycja Trojczak

摘要

A problem of determining real efficiency of the Tall Array method in computing statistics of numeric data is considered. Random arrays of specified numeric types are generated, for which computational time is measured by applying ordinary and Tall Array MATLAB operators in computing statistics of the generated arrays. The main finding is, for two parallel processor workers, the Tall Array method becomes efficient if an array of no fewer than a 10 to 35 million entries is to be processed. The Tall Array minimum and maximum are similarly efficient in processing arrays with double and single precision, both real-valued and complex-valued, and their efficiency is comparable to that of the Tall Array standard deviation. The efficiency threshold of the Tall Array covariance is 1.75 to 5 times less than that of the Tall Array standard deviation. The Tall Array mean is the least efficient for non-integer representation. The Tall Array operators of minimum, maximum, and mean are similarly efficient on unsigned and signed integers. Nevertheless, the Tall Array mean operator appears to be more efficient on signed integers. Moreover, as the number of bits is increased, its efficiency grows, where the efficiency threshold decreases from 20 million entries for int8 down to 5 million entries for int64.