Due to the limitation of web-side processing capabilities, the existing mainstream inference models adopt the client-to-server operation mode, and the emergence of WebAssembly has brought opportunities for the client to run the inference model alone for image recognition, and soon, the browser will fully support the standard. Therefore, on the basis of image recognition, this paper runs the mainstream inference model on the client and server side, and then compares the inference rate of different inference models at different batch sizes and the inference efficiency of the same inference model on the client and server side, conducts a large number of comparative experiments to record the experimental results, and then uses the method of averaging the experimental data to obtain a set of reliable experimental conclusions. Experimental results show that the performance of server-side TensorRT is better than other models under the same configuration conditions, ONNXRuntime is better than PyTorch in small batches, and PyTorch's performance improvement in accuracy has more room for optimization. The results of web-side experiments show that the inference model has a large difference in GPU usage, ONNXRuntime shows a greater performance advantage in the efficiency of GPU inference, while TensorRT will show better performance with the increase of batch. Compared with server-side inference, web-side inference shows stronger stability because it relies less on network transmission, so it can be preferred to direct inference when the batch is small, and server inference is preferred when the client configuration does not meet the conditions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Comparative Study of Inference Models Based on Image Recognition

  • Deling Zhao

摘要

Due to the limitation of web-side processing capabilities, the existing mainstream inference models adopt the client-to-server operation mode, and the emergence of WebAssembly has brought opportunities for the client to run the inference model alone for image recognition, and soon, the browser will fully support the standard. Therefore, on the basis of image recognition, this paper runs the mainstream inference model on the client and server side, and then compares the inference rate of different inference models at different batch sizes and the inference efficiency of the same inference model on the client and server side, conducts a large number of comparative experiments to record the experimental results, and then uses the method of averaging the experimental data to obtain a set of reliable experimental conclusions. Experimental results show that the performance of server-side TensorRT is better than other models under the same configuration conditions, ONNXRuntime is better than PyTorch in small batches, and PyTorch's performance improvement in accuracy has more room for optimization. The results of web-side experiments show that the inference model has a large difference in GPU usage, ONNXRuntime shows a greater performance advantage in the efficiency of GPU inference, while TensorRT will show better performance with the increase of batch. Compared with server-side inference, web-side inference shows stronger stability because it relies less on network transmission, so it can be preferred to direct inference when the batch is small, and server inference is preferred when the client configuration does not meet the conditions.