In power systems, voltage transformers are used as the main data source for voltage measurement, and they show high accuracy in industrial frequency voltage measurement. However, in the face of the increasing harmonic pollution problem in the power grid, the traditional voltage transformer is difficult to provide the required accurate data. To address this challenge, this paper develops a corresponding online monitoring device based on FPGA+ARM architecture to realize real-time monitoring of grid voltage. Firstly, we complete the hardware fabrication, software design and upper computer interface development of the grid wideband voltage online monitoring device; then we adopt the high-precision black-box inverse model as the computational model, and finally, we build a wide-band voltage test platform to carry out the high-harmonic test in order to verify the accuracy and real-time performance of the device developed in this paper. With the proposed broadband voltage online measurement device and its corresponding black-box inverse model compensation method, the average TVE value of the device’s inverted primary voltage in the frequency range of 4 kHz is 2.48%, while the average TVE value of the converted voltage is 23.36%, which is an improvement of 84.08% in accuracy.