<p>Rice blast, caused by the fungus <i>Magnaporthe oryzae</i>, is one of the most devastating diseases of rice, responsible for an estimated global crop loss of 4.33%. Although breeding cultivars resistant to blast is laborious, it is the most effective and sustainable way to mitigate its impact on global rice production. Breeders use the Universal Blast Nursery (UBN) to evaluate thousands of breeding lines for blast resistance in a year to make breeding decisions. These evaluations are visual and subjective making them relatively less reliable and accurate than desired for consistent and reproducible breeding decisions. This paper presents an image-based estimation of blast severity using canopy images representing the entire breeding line and deep-learning neural networks. While countless studies have reported severity estimation using single-leaf images, deploying such techniques is ineffective with canopy images from UBN. This study was conducted in two phases. In the first phase of the study, a relatively shallow model was able to classify the images into “susceptible” and “resistant” lines with an accuracy of 96.67%. Upon observing the misclassified images, higher accuracy was obtained from extracting simple feature attributes such as biomass rather than lesion and other relevant disease symptoms. A “partially susceptible” category was included in the second phase, to improve the model. Despite the reduction in accuracy, the models trained with the progressive resizing strategy were able to extract the smaller intended features. In phase two, the model achieved an accuracy of 87.78%.</p>