IEEE Robotics & Automation Magazine - September 2013 - 59
extract leaves from ToF data and infrared-intensity images.
The segmentation is fully automatic and is based on a novel
depth-segmentation algorithm, which can be applied to
sparse or noisy depth data and can cope with curved surfaces.
Another difference is that leaf models are fitted explicitly,
which allows us to localize grasping points.
The proposed system for automated plant probing is
related to vision-based robotic systems for fruit and vegetable
picking that have been proposed in the past for the automation
of harvesting tasks [6]. Often these systems first process and
segment the data to identify and represent the target. Based on
this representation, a robot action (e.g., cutting or grasping) is
executed. The image-processing task is often eased by fixing
the environment in a specific manner. For example, in a fruitdetachment system developed by Feng et al. [20] in 2008,
strawberries were grown on a uniformly colored surface to
simplify image segmentation. In our system, the environment
is less constrained, and the proposed computer-vision system
is thus more complex. Furthermore, a new robotic application,
i.e., the automatic sampling of leaves with a specific cutting
tool, was introduced and explored. To the authors' knowledge,
this is the first time that an active vision approach using ToF
depth has been applied to robotized plant measuring.
The method is based on several assumptions: 1) the
boundaries of leaves are visible in the infrared-intensity
image, 2) the leaf surfaces can be modeled by a basic quadratic function, 3) leaves of a specific plant type can be
described by a common 2-D contour, 4) leaves are large
enough to be analyzed with a ToF camera, and 5) the leaves
are static during probing. These assumptions may be violated
under certain conditions, but nevertheless we expect the
method to be applicable to many different types of plants,
given a controlled environment.
In conclusion, we tackled a quite complex task that
required the extraction of task-relevant plant parameters from
plant images using a multistage algorithm as well as the difficult problem of the actual execution of the robot motion
toward the plant. The automation of plant probing has a
potentially wide range of applications both in the agricultural
industry, in which certain (currently manual) tasks must be
repeatedly executed for multiple plants, and in botanic experimentation, e.g., for phenotyping, in which leaf sample discs
are commonly used to analyze plant development to determine the genetic factors that control growth.
Acknowledgments
This research was partially funded by EU GARNICS project
FP7-247947, project PAU+ (DPI2011-27510), and Grup consolidat SGR155. B. Dellen acknowledges support from the
Spanish Ministry of Science and Innovation through a
Ramon y Cajal program. S. Foix was supported by a Ph.D. fellowship from CSIC's JAE program.
References
[1] A. Kolb, E. Barth, and R. Koch, "ToF-sensors: New dimensions for realism
and interactivity," in Proc. IEEE CVPR Workshop, 2008, vols. 1-3, pp. 1518-1523.
[2] A. Nealen, M. Muller, R. Keiser, E. Boxerman, and M. Carlson, "Physically
based deformable models in computer graphics," Comput. Graphics Forum,
vol. 25, no. 4, pp. 809-836, 2006.
[3] R. B. Rusu, A. Holzbach, R. Diankov, G. Bradski, and M. Beetz, "Perception
for mobile manipulation and grasping using active stereo," in Proc. 9th IEEERAS Int. Conf. Humanoid Robots, 2009, pp. 632-638.
[4] T. Fourcaud, X. Zhang, A. Stokes, H. Lambers, and C. KÖner, "Plant
growth modelling and applications: The increasing importance of plant architecture in growth models," in Proc. Ann. Botany, 2008, pp. 1053-1063.
[5] E. J. Van Henten, B. A. J. Van Tuijl, G.-J. Hoogakker, M. J. Van Der Weerd, J.
Hemming, J. G. Kornet, and J. Bontsema, "An autonomous robot for de-leafing
cucumber plants grown in a high-wire cultivation system," Biosyst. Eng., vol. 94,
no. 3, pp. 317-323, 2006.
[6] T. Grift, Q. Zhang, N. Kondo, and K. C. Ting, "A review of automation and
robotics for the bio-industry," J. Biomech. Eng., vol. 1, no. 1, pp. 37-54, 2008.
[7] C.-H. Teng, Y.-T. Kuo, and Y.-S. Chen, "Leaf segmentation, classification,
and three-dimensional recovery from a few images with close viewpoints,"
Optical Eng., vol. 50, no. 3, 037003, pp. 1-12, 2011.
[8] L. Quan, P. Tan, G. Zeng, L. Yuan, J. Wang, and S. B. Kang, "Image based
plant modelling," ACM Siggraph, vol. 25, no. 3, pp. 599-604, 2006.
[9] Y. Song, R. Wilson, R. Edmondson, and N. Parsons, "Surface modelling of
plants from stereo images," in Proc. 6th IEEE Int. Conf. 3D Digital Imaging
Modelling, 2007, pp. 312-319.
[10] G. Taylor and L. Kleeman, "Robust range data segmentation using geometric primitives for robotic applications," in Proc. 5th IASTED Int. Conf. Signal Image Processing, 2003, pp. 1-6.
[11] B. I. Loch, J. A. Belward, and J. S. Hanan, "Application of surface fitting
techniques for the representation of leaf surfaces," in Proc. MODSIM Int.
Congr. Modelling Simulation, 2005, pp. 1272-1278.
[12] R. Klose, J. Penlington, and A. Ruckelshausen, "Usability study of 3D
time-of-flight cameras for automatic plant phenotyping," in Proc. Workshop
Computer Image Analysis Agriculture, 2009, pp. 93-105.
[13] S. Foix, G. Alenyà, J. Andrade-Cetto, and C. Torras, "Object modeling
using a ToF camera under an uncertainty reduction approach," in Proc. IEEE
Int. Conf. Robotics Automation, 2010, pp. 1306-1312.
[14] F. F. Khalil and P. Payeur, "Dexterous robotic manipulation of deformable
objects with multi-sensory feedback-a review," in Robot Manipulators Trends
and Development, A. Jimenez and B. M. A. Hadithi, Eds. 2010, pp. 587-619.
[15] G. Alenyà, B. Dellen, and C. Torras, "3D modelling of leaves from color
and ToF data for robotized plant measuring," in Proc. IEEE Int. Conf. Robotics
Automation, 2011, pp. 3408-3414.
[16] B. Biskup, H. Scharr, A. Fischbach, A. Wiese-Klinkenberg, U. Schurr, and
A. Walter, "Diel growth cycle of isolated leaf discs analyzed with a novel, highthroughput three-dimensional imaging method is identical to that of intact
leaves," Plant Physiol., vol. 149, no. 3, pp. 1452-1461, Mar. 2009.
[17] S. Foix, G. Alenyà, and C. Torras, "Lock-in time-of-flight cameras: A survey," IEEE Sensors J., vol. 11, no. 9, pp. 1917-1926, 2011.
[18] S. Fuchs and S. May, "Calibration and registration for precise surface
reconstruction with time of flight cameras," Int. J. Int. Syst. Tech. App., vol. 5,
nos. 3-4, pp. 274-284, 2008.
[19] P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image
segmentation," Int. J. Comput. Vision, vol. 59, no. 2, pp. 167-181, 2004.
[20] G. Feng, C. Qizin, and M. Masateru, "Fruit detachment and classification
method for strawberry harvesting robot," Int. J. Adv. Robot. Syst., vol. 5, no. 1,
pp. 41-48, 2008.
Guillem Alenyà, Institut de Robotica i Informatica Industrial
(CSIC-UPC), Barcelona, Spain. E-mail: galenya@iri.upc.edu.
Babette Dellen, Institut de Robotica i Informatica Industrial
(CSIC-UPC), Barcelona, Spain. E-mail: bdellen@iri.upc.edu.
Sergi Foix, Institut de Robotica i Informatica Industrial (CSICUPC), Barcelona, Spain. E-mail: sfoix@iri.upc.edu.
Carme Torras, Institut de Robotica i Informatica Industrial
(CSIC-UPC), Barcelona, Spain. E-mail: torras@iri.upc.edu.
september 2013
*
IEEE ROBOTICS & AUTOMATION MAGAZINE
*
59
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2013
IEEE Robotics & Automation Magazine - September 2013 - Cover1
IEEE Robotics & Automation Magazine - September 2013 - Cover2
IEEE Robotics & Automation Magazine - September 2013 - 1
IEEE Robotics & Automation Magazine - September 2013 - 2
IEEE Robotics & Automation Magazine - September 2013 - 3
IEEE Robotics & Automation Magazine - September 2013 - 4
IEEE Robotics & Automation Magazine - September 2013 - 5
IEEE Robotics & Automation Magazine - September 2013 - 6
IEEE Robotics & Automation Magazine - September 2013 - 7
IEEE Robotics & Automation Magazine - September 2013 - 8
IEEE Robotics & Automation Magazine - September 2013 - 9
IEEE Robotics & Automation Magazine - September 2013 - 10
IEEE Robotics & Automation Magazine - September 2013 - 11
IEEE Robotics & Automation Magazine - September 2013 - 12
IEEE Robotics & Automation Magazine - September 2013 - 13
IEEE Robotics & Automation Magazine - September 2013 - 14
IEEE Robotics & Automation Magazine - September 2013 - 15
IEEE Robotics & Automation Magazine - September 2013 - 16
IEEE Robotics & Automation Magazine - September 2013 - 17
IEEE Robotics & Automation Magazine - September 2013 - 18
IEEE Robotics & Automation Magazine - September 2013 - 19
IEEE Robotics & Automation Magazine - September 2013 - 20
IEEE Robotics & Automation Magazine - September 2013 - 21
IEEE Robotics & Automation Magazine - September 2013 - 22
IEEE Robotics & Automation Magazine - September 2013 - 23
IEEE Robotics & Automation Magazine - September 2013 - 24
IEEE Robotics & Automation Magazine - September 2013 - 25
IEEE Robotics & Automation Magazine - September 2013 - 26
IEEE Robotics & Automation Magazine - September 2013 - 27
IEEE Robotics & Automation Magazine - September 2013 - 28
IEEE Robotics & Automation Magazine - September 2013 - 29
IEEE Robotics & Automation Magazine - September 2013 - 30
IEEE Robotics & Automation Magazine - September 2013 - 31
IEEE Robotics & Automation Magazine - September 2013 - 32
IEEE Robotics & Automation Magazine - September 2013 - 33
IEEE Robotics & Automation Magazine - September 2013 - 34
IEEE Robotics & Automation Magazine - September 2013 - 35
IEEE Robotics & Automation Magazine - September 2013 - 36
IEEE Robotics & Automation Magazine - September 2013 - 37
IEEE Robotics & Automation Magazine - September 2013 - 38
IEEE Robotics & Automation Magazine - September 2013 - 39
IEEE Robotics & Automation Magazine - September 2013 - 40
IEEE Robotics & Automation Magazine - September 2013 - 41
IEEE Robotics & Automation Magazine - September 2013 - 42
IEEE Robotics & Automation Magazine - September 2013 - 43
IEEE Robotics & Automation Magazine - September 2013 - 44
IEEE Robotics & Automation Magazine - September 2013 - 45
IEEE Robotics & Automation Magazine - September 2013 - 46
IEEE Robotics & Automation Magazine - September 2013 - 47
IEEE Robotics & Automation Magazine - September 2013 - 48
IEEE Robotics & Automation Magazine - September 2013 - 49
IEEE Robotics & Automation Magazine - September 2013 - 50
IEEE Robotics & Automation Magazine - September 2013 - 51
IEEE Robotics & Automation Magazine - September 2013 - 52
IEEE Robotics & Automation Magazine - September 2013 - 53
IEEE Robotics & Automation Magazine - September 2013 - 54
IEEE Robotics & Automation Magazine - September 2013 - 55
IEEE Robotics & Automation Magazine - September 2013 - 56
IEEE Robotics & Automation Magazine - September 2013 - 57
IEEE Robotics & Automation Magazine - September 2013 - 58
IEEE Robotics & Automation Magazine - September 2013 - 59
IEEE Robotics & Automation Magazine - September 2013 - 60
IEEE Robotics & Automation Magazine - September 2013 - 61
IEEE Robotics & Automation Magazine - September 2013 - 62
IEEE Robotics & Automation Magazine - September 2013 - 63
IEEE Robotics & Automation Magazine - September 2013 - 64
IEEE Robotics & Automation Magazine - September 2013 - 65
IEEE Robotics & Automation Magazine - September 2013 - 66
IEEE Robotics & Automation Magazine - September 2013 - 67
IEEE Robotics & Automation Magazine - September 2013 - 68
IEEE Robotics & Automation Magazine - September 2013 - 69
IEEE Robotics & Automation Magazine - September 2013 - 70
IEEE Robotics & Automation Magazine - September 2013 - 71
IEEE Robotics & Automation Magazine - September 2013 - 72
IEEE Robotics & Automation Magazine - September 2013 - 73
IEEE Robotics & Automation Magazine - September 2013 - 74
IEEE Robotics & Automation Magazine - September 2013 - 75
IEEE Robotics & Automation Magazine - September 2013 - 76
IEEE Robotics & Automation Magazine - September 2013 - 77
IEEE Robotics & Automation Magazine - September 2013 - 78
IEEE Robotics & Automation Magazine - September 2013 - 79
IEEE Robotics & Automation Magazine - September 2013 - 80
IEEE Robotics & Automation Magazine - September 2013 - 81
IEEE Robotics & Automation Magazine - September 2013 - 82
IEEE Robotics & Automation Magazine - September 2013 - 83
IEEE Robotics & Automation Magazine - September 2013 - 84
IEEE Robotics & Automation Magazine - September 2013 - 85
IEEE Robotics & Automation Magazine - September 2013 - 86
IEEE Robotics & Automation Magazine - September 2013 - 87
IEEE Robotics & Automation Magazine - September 2013 - 88
IEEE Robotics & Automation Magazine - September 2013 - 89
IEEE Robotics & Automation Magazine - September 2013 - 90
IEEE Robotics & Automation Magazine - September 2013 - 91
IEEE Robotics & Automation Magazine - September 2013 - 92
IEEE Robotics & Automation Magazine - September 2013 - 93
IEEE Robotics & Automation Magazine - September 2013 - 94
IEEE Robotics & Automation Magazine - September 2013 - 95
IEEE Robotics & Automation Magazine - September 2013 - 96
IEEE Robotics & Automation Magazine - September 2013 - 97
IEEE Robotics & Automation Magazine - September 2013 - 98
IEEE Robotics & Automation Magazine - September 2013 - 99
IEEE Robotics & Automation Magazine - September 2013 - 100
IEEE Robotics & Automation Magazine - September 2013 - 101
IEEE Robotics & Automation Magazine - September 2013 - 102
IEEE Robotics & Automation Magazine - September 2013 - 103
IEEE Robotics & Automation Magazine - September 2013 - 104
IEEE Robotics & Automation Magazine - September 2013 - Cover3
IEEE Robotics & Automation Magazine - September 2013 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com