Automatic sweet pepper detection based on point cloud images using subtractive clustering

Authors

  • Xiaokang Zhao Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education, Jiangnan University, Wuxi 214122, China
  • Hao Li Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education, Jiangnan University, Wuxi, 214122, China
  • Qibing Zhu Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education, Jiangnan University, Wuxi, 214122, China
  • Min Huang Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education, Jiangnan University, Wuxi, 214122, China
  • Ya Guo Key Laboratory of Advanced Process Control for Light Industry of Ministry of Education, Jiangnan University, Wuxi, 214122, China
  • Jianwei Qin USDA/ARS Environmental Microbial and Food Safety Laboratory, Beltsville Agricultural Research Center, Bldg., 303, BARC-East, 10300 Baltimore Ave., MD 20705-2350, USA

Keywords:

sweet pepper detection, point cloud, subtractive clustering, computer vision

Abstract

Automatic identification and detection of fruit on trees by machine vision is the basis of developing automatic harvesting robots in agriculture. The occlusion of branches, leaves and other fruits in canopy images will affect the accuracy of fruit detection. To provide a scientific and reliable technical guidance for fruit harvesting robots, a method using point cloud images was proposed in this study to detect red fruits to overcome the impact of occlusion on detection. Firstly, the fruit regions were segmented from a tree’s point cloud by applying the color threshold of red and green. Then, the noise in fruit point clouds was removed with sparse outlier removal. Finally, the point cloud of each fruit was detected and counted based on the subtractive clustering algorithm. For the sweet pepper dataset, the true positive rate (TPR) is 90.69% and the false positive rate (FPR) is 6.97% for all fruits that are at least partially visible in the scene. Keywords: sweet pepper detection, point cloud, subtractive clustering, computer vision DOI: 10.25165/j.ijabe.20201303.5460 Citation: Zhao X K, Li H, Zhu Q B, Huang M, Guo Y, Qin J W. Automatic sweet pepper detection based on point cloud images using subtractive clustering. Int J Agric & Biol Eng, 2020; 13(3): 154–160.

References

Shamshiri R R, Hameed I A, Karkee M, Weltzien C. Robotic harvesting of fruiting vegetables: A simulation approach in V-REP, ROS and MATLAB. In: Kulshreshtha S N (Ed.). Automation in Agriculture-Securing Food Supplies for Future Generations, London: InTechOpen, 2018; pp.81–105.

Shamshiri R R, Weltzien C, Hameed I A, Yule I J, Grift T E, Balasundram S K, et al. Research and development in agricultural robotics: A perspective of digital farming. Int J Agric & Biol Eng, 2018; 11(4): 1–14.

Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K. Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture, 2015; 116: 8–19.

Edan Y, Rogozin D, Flash T, Miles G E. Robotic melon harvesting. IEEE Transactions on Robotics and Automation, 2000; 16(6): 831–835.

Lu Q, Cai J R, Liu B, Deng L, Zhang Y J. Identification of fruit and branch in natural scenes for citrus harvesting robot using machine vision and support vector machine. Int J Agric & Biol Eng, 2014; 7(2): 115–121.

Tabb A L, Peterson D L, Parker J. Segmentation of Apple fruit from video via background modeling. In: ASABE Annual International Meeting Presentation. St. Joseph: ASABE, 2006; pp.60–63.

Kurtulmus F, Lee W S, Vardar A. Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network. Precision Agriculture, 2014; 15: 57–79.

Khoshroo A, Arefi A, Khodaei J. Detection of red tomato on plants using image processing techniques. Agricultural Communications, 2014; 2(4): 9–15.

Kuang H L, Liu C R, Chan L L H, Yan H. Multi-class fruit detection based on image region selection and improved object proposals. Neurocomputing, 2018; 283: 241–255.

Sa I, Ge Z Y, Dayoub F, Upcroft B, Perez T, McCool C. Deepfruits: A fruit detection system using deep neural networks. Sensors, 2016; 16(8): 1222.

Shamshiri R R, Hameed I A, Pitonakova L, Weltzien C, Balasundram S K, Yule I J. Simulation software and virtual environments for acceleration of agricultural robotics: Features highlights and performance comparison. Int J Agric & Biol Eng, 2018; 11(4): 15–31.

Wang Q, Nuske S, Bergerman M, Singh S. Automated crop yield estimation for apple orchards. Experimental Robotics, 2013; 88: 745-758.

Xiang R, Jiang H, Ying Y. Recognition of clustered tomatoes based on binocular stereo vision. Computers and Electronics in Agriculture, 2014; 106: 75–90.

Wei X, He J C, Ye D P, Jie D F. Navel orange maturity classification by multispectral indexes based on hyperspectral diffuse transmittance imaging. Journal of Food Quality, 2017; 7: 1–7.

Underwood J P, Hung C, Whelan B, Sukkarieh S. Mapping almond orchard canopy volume, flowers, fruit and yield using lidar and vision sensors. Computers and Electronics in Agriculture, 2016; 130: 83–96.

Stein M, Bargoti S, Underwood J. Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors, 2016; 16(11): 1915.

Gan H, Lee W S, Alchanatis V, Ehsani R, Schueller J K. Immature green citrus fruit detection using color and thermal images. Computers and Electronics in Agriculture, 2018; 152: 117–125.

Gongal A, Silwal A, Amatya S, Karkee M, Zhang Q, Lewis K. Apple crop-load estimation with over-the-row machine vision system. Computers and Electronics in Agriculture, 2016; 120: 26–35.

Tao Y T, Zhou J. Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking. Computers and Electronics in Agriculture, 2017; 142(A): 388–396.

Qureshi W S, Payne A, Walsh K B, Linker R, Cohen O, Dailey M N. Machine vision for counting fruit on mango tree canopies. Precision Agriculture, 2017; 18: 224–244.

Sa I, Lehnert C, English A, Mccool C, Dayoub F, Upcroft B, et al. Peduncle detection of sweet pepper for autonomous crop harvesting— combined color and 3-D information. IEEE Robotics and Automation Letters, 2017; 2(2): 765–772.

Lehnert C, Sa I, McCool C, Upcroft B, Perez T. Sweet pepper pose detection and grasping for automated crop harvesting. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). Stockolm: IEEE, 2016; pp.2428–2434.

Rusu R B, Cousins S. 3D is here: Point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and Automation. Shanghai: IEEE, 2011; pp.1–4.

Rusu R B, Marton Z C, Blodow N, Dolha M, Beetz M. Towards 3d point cloud based object maps for household environments. Robotics and Autonomous Systems, 2008; 56(11): 927–941.

Fernandez D, Parra I, Sotelo M A, Revenga P, Gavilan M. 3D candidate selection method for pedestrian detection on non-planar roads. In: 2007 IEEE Intelligent Vehicles Symposium. Istanbul: IEEE, 2007; pp.1162–1167.

Wu G, Zhu Q B, Huang M, Guo Y, Qin J W. Automatic recognition of juicy peaches on trees based on 3D contour features and colour data. Biosystems Engineering, 2019; 188: 1–13.

Downloads

Published

2020-06-08

How to Cite

Zhao, X., Li, H., Zhu, Q., Huang, M., Guo, Y., & Qin, J. (2020). Automatic sweet pepper detection based on point cloud images using subtractive clustering. International Journal of Agricultural and Biological Engineering, 13(3), 154–160. Retrieved from https://www.ijabe.migration.pkpps06.publicknowledgeproject.org/index.php/ijabe/article/view/5460

Issue

Vol. 13 No. 3 (2020): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed, open access journal, adopting Creative Commons Copyright Notices as follows.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).