Purpose <p>To provide scientific and reliable technical guidance for unmanned agricultural machinery in orchard fruit picking, this study combines Neural Radiance Fields (NeRF) with a semantic segmentation network based on PointNeXt.</p> Methods <p>A method for apple fruit recognition using three-dimensional point clouds is proposed. The NeRF theory is employed to construct a single-tree three-dimensional point cloud model of apple trees. The resulting point cloud model of the trees exhibits detailed representation, thereby supplying a dataset of high accuracy and standards for semantic segmentation experiments. Initially, panoramic videos are captured around the trees to obtain multi-view images of the trees with camera poses. Subsequently, using NeRF, a three-dimensional model of the trees is established and annotated, with the point cloud of the trees labeled into fruit and branches/leaves, thereby creating the dataset required for training PointNeXt. Finally, semantic segmentation experiments are conducted using PointNeXt to segment the three-dimensional point cloud of apple fruits for identification purposes.</p> Results <p>The OA, mAcc, and mIoU of PointNeXt are reported to be 0.9313, 0.9123, and 0.9036. The apple fruit point cloud diameters R<sup>2</sup> and RMSE obtained through PointNeXt semantic segmentation are 0.90458 and 1.97 mm, respectively, while those obtained through PointNet++ spherical fitting are 0.84570 and 2.88 mm, respectively.</p> Conclusion <p>The experimental results demonstrate that this research method effectively identifies fruits within entire trees, addressing the issue of apple point cloud misidentification due to complex environmental occlusion. This approach enhances the accuracy of apple fruit recognition, accurately reflecting the scale and shape of fruits, thereby providing technical support for unmanned orchard machinery in apple picking operations.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Recognition of Apple Fruit at Maturity Based on Neural Radiance Field and PointNeXt

  • Jianhang Wang,
  • Huiyan Wang,
  • Binxiao Liu,
  • Jinliang Gong,
  • Yanfei Zhang

摘要

Purpose

To provide scientific and reliable technical guidance for unmanned agricultural machinery in orchard fruit picking, this study combines Neural Radiance Fields (NeRF) with a semantic segmentation network based on PointNeXt.

Methods

A method for apple fruit recognition using three-dimensional point clouds is proposed. The NeRF theory is employed to construct a single-tree three-dimensional point cloud model of apple trees. The resulting point cloud model of the trees exhibits detailed representation, thereby supplying a dataset of high accuracy and standards for semantic segmentation experiments. Initially, panoramic videos are captured around the trees to obtain multi-view images of the trees with camera poses. Subsequently, using NeRF, a three-dimensional model of the trees is established and annotated, with the point cloud of the trees labeled into fruit and branches/leaves, thereby creating the dataset required for training PointNeXt. Finally, semantic segmentation experiments are conducted using PointNeXt to segment the three-dimensional point cloud of apple fruits for identification purposes.

Results

The OA, mAcc, and mIoU of PointNeXt are reported to be 0.9313, 0.9123, and 0.9036. The apple fruit point cloud diameters R2 and RMSE obtained through PointNeXt semantic segmentation are 0.90458 and 1.97 mm, respectively, while those obtained through PointNet++ spherical fitting are 0.84570 and 2.88 mm, respectively.

Conclusion

The experimental results demonstrate that this research method effectively identifies fruits within entire trees, addressing the issue of apple point cloud misidentification due to complex environmental occlusion. This approach enhances the accuracy of apple fruit recognition, accurately reflecting the scale and shape of fruits, thereby providing technical support for unmanned orchard machinery in apple picking operations.