Speech recognition is the process of recognizing and understanding human voices, and converting them into textual information. Speech recognition is a complex and important technology, and it is an important research topic. On this basis, a speech recognition method based on embedded technology was proposed. Based on the above research, this article designed and implemented an embedded Japanese translation system. Language recognition technology is a machine that recognizes, understands, and converts language information into textual information. Nowadays, computer dictionaries and sound recognition technology are both advancing. Due to the advancement of chip technology, embedded systems have more functions and speech recognition technology can be embedded into embedded systems. Therefore, embedded speech recognition has become a new development direction. This article explored a Japanese speech recognition and real-time translation system based on convolutional neural networks (CNN). Firstly, it explored how to build a speech recognition system and then constructed a translation system. After that, it constructed an end-to-end speech recognition model. Finally, the superiority of the system in this article was verified through experiments (the accuracy, recall, and F1 mean of the speech recognition and real-time translation system based on the algorithm in this article were higher than or equal to 0.78).