Recognition of Apple Fruit at Maturity Based on Neural Radiance Field and PointNeXt
摘要
To provide scientific and reliable technical guidance for unmanned agricultural machinery in orchard fruit picking, this study combines Neural Radiance Fields (NeRF) with a semantic segmentation network based on PointNeXt.
MethodsA method for apple fruit recognition using three-dimensional point clouds is proposed. The NeRF theory is employed to construct a single-tree three-dimensional point cloud model of apple trees. The resulting point cloud model of the trees exhibits detailed representation, thereby supplying a dataset of high accuracy and standards for semantic segmentation experiments. Initially, panoramic videos are captured around the trees to obtain multi-view images of the trees with camera poses. Subsequently, using NeRF, a three-dimensional model of the trees is established and annotated, with the point cloud of the trees labeled into fruit and branches/leaves, thereby creating the dataset required for training PointNeXt. Finally, semantic segmentation experiments are conducted using PointNeXt to segment the three-dimensional point cloud of apple fruits for identification purposes.
ResultsThe OA, mAcc, and mIoU of PointNeXt are reported to be 0.9313, 0.9123, and 0.9036. The apple fruit point cloud diameters R2 and RMSE obtained through PointNeXt semantic segmentation are 0.90458 and 1.97 mm, respectively, while those obtained through PointNet++ spherical fitting are 0.84570 and 2.88 mm, respectively.
ConclusionThe experimental results demonstrate that this research method effectively identifies fruits within entire trees, addressing the issue of apple point cloud misidentification due to complex environmental occlusion. This approach enhances the accuracy of apple fruit recognition, accurately reflecting the scale and shape of fruits, thereby providing technical support for unmanned orchard machinery in apple picking operations.