A large number of high-strength bolts are used to connect multiple steel members in steel bridge construction. All high-strength bolts are visually inspected one by one to confirm whether they are fastened correctly or not, which requires a large effort for construction workers. This paper proposes an automatic bolt detection method for automatically determining whether a high-strength bolt detected is fastened correctly. In the proposed method, images of multiple high-strength bolts are raster-scanned, and a bolt presence probability map is computed from the high-strength bolt detection windows obtained by the support vector machine based on the histogram of oriented gradient features of an image patch. Since the bolt size is not constant in the image, the exact bolt bounding box is estimated by fuzzy inference using the obtained detection windows and bolt existence probabilities. The experiments were conducted on 90 training images containing 1,411 bolts and 65 validation images containing 1,026 bolts that were acquired in the steel bridge construction sites. Experimental results showed that the proposed method detected high-strength bolts in 0.959 precision, 0.728 recall, and 0.828 F1-score with fuzzy inference. The bolt fastening inspection was performed using SVM or MobileNetV3 as three classes classification of normal installation, abnormal installation, and incorrect marking. Results showed that MobileNetV3 was more accurate than SVM for the bolt fastening inspection in 0.878 precision, 0.848 recall, and 0.881 F1-score.