Immature Green Apple Detection and Sizing in Commercial Orchards using YOLOv8 and Shape Fitting Techniques

Overview

• Detecting and measuring the size of immature green apples (fruitlets) is crucial for predicting yield, managing pests, and making decisions about crop management, harvesting, and marketing.

• Traditional fruit size measurement methods are labor-intensive and time-consuming.

• This study used a state-of-the-art object detection and instance segmentation algorithm (YOLOv8) with 3D point cloud data to accurately determine the size of green apples in a commercial orchard environment.

• Two RGB-D sensors, the Intel RealSense D435i and Microsoft Azure Kinect DK, were used to capture the data.

Plain English Explanation

Growing apples is an important agricultural activity, but farmers face challenges in predicting the size and quantity of their crop. Accurately measuring the size of young, unripe apples (called fruitlets) can help farmers make better decisions about managing their orchards, preparing for harvest, and selling their produce.

Traditional methods of measuring fruit size are complex and time-consuming. This research explored a more efficient solution using advanced computer vision techniques. The researchers used a powerful object detection algorithm called YOLOv8 to identify and segment individual fruitlets in 3D images captured by specialized cameras. By analyzing the shape and size of the fruitlets, the researchers could accurately estimate their dimensions.

Compared to manual measurement, this automated approach was much faster and more precise. The researchers found that the Microsoft Azure Kinect camera produced more accurate size estimates than the Intel RealSense camera. However, the system still struggled with some challenges, like when fruitlets were partially hidden by other fruit on the tree.

Overall, this research demonstrates a promising new way for apple growers to monitor the development of their crop and make more informed decisions throughout the growing season and leading up to harvest.

Technical Explanation

The study employed the YOLOv8 object detection and instance segmentation algorithm to identify and delineate individual immature green apples (fruitlets) in 3D point cloud data. Two RGB-D sensors were used to capture the data:

1. Intel RealSense D435i
2. Microsoft Azure Kinect DK

The YOLOv8 instance segmentation models exhibited strong performance in detecting the green fruitlets, with the YOLOv8m-seg model achieving the highest [email protected] and [email protected] scores of 0.94 and 0.91, respectively.

To estimate the size of the fruitlets, an ellipsoid fitting technique was applied to the 3D point cloud data from the Azure Kinect sensor. This resulted in an RMSE of 2.35 mm, MAE of 1.66 mm, MAPE of 6.15 mm, and an R-squared value of 0.9.

In comparison, the size estimation technique performed better on the Azure Kinect data than the RealSense D435i data. The Azure Kinect achieved an RMSE of 2.35 mm, MAE of 1.66 mm, and R-squared of 0.9, while the RealSense D435i had an RMSE of 9.65 mm, MAE of 7.8 mm, and R-squared of 0.77.

The researchers noted that partial occlusion of the fruitlets, particularly in clusters, caused some error in accurately delineating and sizing the green apples using the YOLOv8-based segmentation technique.

Critical Analysis

The research presents a promising approach for automating the measurement of immature apple size, but there are a few potential limitations and areas for further exploration:

• The study was conducted in a commercial orchard environment, so the performance of the system may vary in different growing conditions or with different apple varieties.
• The accuracy of the size estimation was still affected by occlusion, particularly for fruitlets in clusters. More advanced segmentation techniques or multi-view approaches may be needed to address this challenge.
• The study compared two RGB-D sensors, but there may be other camera technologies or configurations that could further improve the accuracy and robustness of the size estimation.
• The research did not explore the potential for this system to be integrated with other precision agriculture technologies, such as autonomous sprayers or robotic harvesters, which could enhance its real-world applicability.

Overall, this research demonstrates the potential for computer vision and 3D sensing to revolutionize fruit size measurement, but continued development and testing will be necessary to make the technology truly practical and scalable for commercial apple growers.

Conclusion

This study showcases a novel approach to accurately measuring the size of immature green apples (fruitlets) in a commercial orchard environment. By combining state-of-the-art object detection and instance segmentation techniques with 3D point cloud data, the researchers were able to achieve highly accurate size estimates, outperforming traditional manual measurement methods.

The ability to automatically and precisely monitor fruit development throughout the growing season has significant implications for apple growers. It can help them better predict yields, optimize crop management, and streamline harvesting and post-harvest logistics. As this technology continues to evolve, it could become an invaluable tool for increasing the efficiency and productivity of apple orchards around the world.