We consider the issues of performance analysis and optimization of deep learning inference on RISC-V devices using the OpenVINO toolkit. Inference of three transformers for image classification is analyzed. The Structure of Heterogeneous Library (SHL) is integrated to the OpenVINO toolkit to accelerate fully connected layers. We parallelize calculations at the level of OpenVINO using OpenMP. Inference profiling using the modified version showed speedup from \(\sim \) 3.5 to 6.5 times depending on the model compared to the original OpenVINO version. Comparison with the Apache TVM tensor compiler demonstrated that the developed implementation is \(\sim \) 2.27 and 2.69 times inferior on the transformers, where initially fully connected layers took more than 40% of the inference time, and by \(\sim \) 13 times, where convolutional layers dominate (more than 50%). The obtained results confirm the prospects for further optimization of another operations. The paper provides a general scheme of integrating third-party functions, applicable to other operations. Currently, the modifications have been merged to the main branch of the OpenVINO repository, and the community is integrating the kernels of the SHL library for other critical operations.