In application of robots in harvesting of agricultural and horticultural crops, a common problem is the occurrence of overlapping objects in a scene. Typically, a partially obscured object must be conclusively identified and extracted before it can be picked by the robot. Morphological operations can be used to smooth edges of a region, force shapes onto region edges, count regions and estimate the size of regions. Morphological operators typically use set theories of intersection and union. The case of separating overlapping flowers is presented in this paper. Thresholding, followed by sequential erosion and dilation were successfully applied to separate overlapping flowers in a series of images.

Keywords: image analysis, morphological operations, robotic harvesting

The mechanization of horticultural harvesting operations is a very important aspect of modern commercial and subsistence horticulture. Mechanization reduces the drudgery of farm work and can reduce labour costs. The timeliness of operations is also enhanced, reducing timeliness costs while ensuring prompt delivery of produce to the markets. Various manually operated equipment are currently available for harvesting different horticultural produce. However, an increasing trend in industrial and agricultural production processes is the use of automatic systems such as robots. This has been made possible because of the recent considerable advances in computer technology. Robotic intelligence has been enhanced through computer vision technology.

Computer vision, also referred to as digital image processing (DIP), studies the underlying principles of human visual perception and attempts to provide a computer-camera system with visual capabilities.¹ A number of applications of DIP in agriculture and horticulture have been realized. The applications can generally be classified into image analysis, quality inspection and robotic vision.² Computer vision based systems for weed control,^3,4 rose sorting,⁵ apple sorting Rehkugler and Throop,⁶ and plant identification⁷ are among the numerous applications that have been attempted in agriculture. Also, Chaugule and Mali⁸ reported the separation of occluded seeds using image analysis techniques.

The results presented in this paper relate to developing a computer vision system to be applied in robotic harvesting of Gerbera jamesonii cut flower. The flower has an indeterminate flowering nature. This implies that at harvesting the plant will typically have flowers at different development stages. The grower has to select mature plants from immature ones and this is largely a subjective decision. In a quality conscious market, erroneous selection of plants can lead to loss of revenue. It is therefore important that an intelligent machine be developed that can identify mature plants in the production environment and pave the way for the subsequent automatic harvesting. A common problem that the robotic harvesting machine would encounter is the issue of overlapping images in a typical scene. Such overlapping images have to be identified and separated before harvesting can take place.

Objectives of the study

The goal of the study was to make it possible to identify mature gerbera flowers using a machine vision system, so as to enable future robotic harvesting. The specific objective of the study was to apply image processing techniques to separate overlapping image objects in a scene in order to facilitate accurate harvesting

Theory of morphological operations

Morphology is a theory for the analysis of spatial structures. Mathematical morphology provides a set of extremely useful operations that enable us to modify or describe the shape of objects. Morphological operations can be defined on regions and gray value images Steger et al.,⁹ Segmentation results often contain unwanted noisy parts. Furthermore, sometimes the segmentation will contain parts in which the shape of the object of interest has been distorted e.g. because of reflections. Therefore, often we need to the shape of segmented regions to obtain the desired results.

All region morphology operations can be defined in terms of six very simple operations; union, intersection, difference, complement, translation and transposition. Steger et al.,⁹ resents the formulae and algorithms for these simple but important operations. These six building blocks are the basis of all morphological operations.

There are typically two regions involved in a morphological operation. The first one is the region we want to process, which will be denoted by R. The other region has a special meaning. It is called the structuring element (or mask) and will be denoted by S. The structuring element is the means by which one can express the shape of interest. Minkowski addition is a common morphological operation which is used to enlarge regions.¹ It involves moving a transposed structuring element around the plane (of the region R). Whenever the translated transposed restructuring element and the region have at least one point in common, the translated reference point is copied into the output.

A Minkowski operation which returns all translated reference points for which the structuring element itself has at least one point in common with the region is called a dilation. The Minkowski addition and dilation enlarge the input region. This can be used to merge separate parts of a region into a single part and thus obtain the connected components of objects. The dilation is also very useful for constructing regions of interest (ROIs) based on regions that were extracted from the image.

The second type of morphological operation is the Minkowski substraction. It is an intersection operation. Hence one can still think about moving the regions R by all vectors s from S. However, now the points must be contained in all translated regions (instead of at least one translated region). For the Minkowski subtraction the structuring element must be completely within the region. This operation is called an erosion. The Minkowski subtraction and erosion shrink the output region. This can be used to separate objects that are attached to one another.  According to Steger et al. (2015), morphological operations can be extended to grey value images.

Image aquisition and storage

Images acquired using a digital camera (Olympus C820-L) were downloaded into the hard disk of computer (Trinology TCAT-513) with an Intel Pentium 133 processor, 128 MB RAM and UW-SCSI hard disk. The images were stored as 1024*768 pixel RGB images in a TIFF compression format. The computer was running on Windows NT 4.0 operating system. Halcon/C++ (MVTec GmbH, Munich) machine vision applications software (source code) Version 5.2 was used for the development of digital image processing program.

Image processing and morphological operations

A program was developed in Halcon/C++ to carry out the major operations for digital image processing. These include: checking of object (flower) presence, binarization of the image, preprocessing, segmentation and feature extraction.¹⁰

After thresholding and segmentation of the images, morphological filters for dilation and erosion were applied on the images. Dilation adds pixels on a region by using a structuring element. The boundary of the region gets smoothed in the process and the area of the region is enlarged. Furthermore disconnected regions can be merged. Such regions, however, remain logically distinct regions. Erosion removes pixels from the region border using a structuring element, thereby smoothing the boundary of the region in the process and reducing the area of the region. Further connected regions may be split. This enabled the removal of frayed region borders and tiny regions.¹¹

Erosion followed by dilation (also called opening) was applied on the binary images. This involves eroding the input region using a small circular mask, getting the connected components after the erosion and then dilating these components.  The smaller regions are discarded after the procedure and the remaining region is subjected to the opening segmentation procedure again by using a large circular mask. The largest inner circle of the resulting region is then obtained and this is used to select the single flower of the region. The assumption here is that the flower of interest will probably be the one nearest to the camera lens during image acquisition and hence the largest in the image. This procedure was implemented on the Halcon/C++ machine vision software platform. The flow chart for the overlapping flowers separation procedure is shown below;

The results of the separation procedure is shown in the Figure 2 below. These separated images confirm that opening and closing operations using different masks of morphological operators can be successfully applied in separating overlapping images. It was easy to isolate the two flowers and then separate the overlapping ones because of the distinct colour difference between the object (flower) and the background. When the background and the object are identical, the separation must involve other procedures for edge detection in addition to morphological operations. According to Szeliski (2022), there are now reliable techniques for accurately computing a 3D model of an environment from thousands of partially overlapping photographs. This work could thus be seen as a precursor in the development of separation techniques for overlapping images.

Figure 1 Flow chart for overlapping images separation procedure.

Figure 2 Separation of multiple Gerberra jamesonii flowers. The output region obtained is not exactly the same as the whole flower. It is possible to process the region to obtain the whole flower.

Morphological operations of dilation and erosion can be applied to separate image objects. From the initial image objects from the scene, segmentation was carried out first. This is then followed by domain reduction to locate the entire image. The main image in the picture is selected by choosing the object with the largest inner circle. The selected part can then be enlarged using a series of erosion masks in order to recover the whole image.

None.

There is no conflicts of interest.

1. Low A. Introductory computer vision and image processing. McGraw–Hill. London; 1991.
2. Kranzler GA. Applying digital image processing in agricultural engineering. Agricultural Engineering. 1985;66:11–13.
3. Brivot R, JA Marchant. Segmentation of plants and weeds for a precision crop protection robot using infra–red images. IEEE Proc VI. Image Signal processing. 1996;143(2).
4. Lee WS, Slaughter DC, Giles DK. Robotic weed control system for tomatoes using machine vision system and precision chemical application. ASAE –Paper.
5. Steinmetz VMJ. Delwiche R Evans and DKGiles. Sorting cut roses with machine vision. Trans Of the ASAE. 1994;41(1549–1555).
6. Low A. Introductory computer vision and image processing. McGraw–Hill. London; 1991.
7. Rath T. Computerbildanalyse im Gratenbau. Gaertenboerse. 1994;94(30):1483–1486.
8. Chaugule A, Mali SN. Digital separation of occluded seeds using image analysis. International Journal of Computer Applications. 2015;123(4).
9. Steger C, Ulrich M, Wiedemann C. Machine vision algorithms and applications. Wiley–VCH. 2015.
10. Ajwang PO. Computer vision for detection of cut flower maturity. MSc. thesis, Leibniz University of Hannover. Germany; 2000.
11. Szeliski R. Computer vision – Algorithms and applications. 2nd ed. Springer; 2022.