In this research, a robot pumpkinharvester was developed to solve the problem of lacking of labor force for harvesting pumpkins. A robot hand-eye system was developed to find and harvest the pumpkins. a color camera was utilized to capture the image from afield and a deep neural network (DNN) was adopted to find the fruits. The detection results were utilized to control the robot arm and robot hand to harvest the pumpkin. In this paper, the design of the robot hand will be introduced. The geometry size of the pumpkin was measured in field. The hand was developed to match the size of the pumpkin. In addition, the dimensions of each links were optimized in order reduced the force for harvesting the pumpkin. The torque requirement of the hand for harvesting the pumpkin fruit was calculated by using a dynamics simulation method.

Keywords:pumpkin, harvester, robot hand, kinematical model, dynamics simulation

More than 40% of pumpkins are planted in Hokkaido, Japan. In recent years, the planting area of pumpkin decreased year by year for the shortage of farmers for harvesting. We are going to develop harvest robot system to help the farmers keep the current planting scale in Japan. However, the complex environment in which pumpkin are grown and various shapes and sizes, which make it is difficult for robot hand to automatically pick pumpkin.¹ It's indispensable that building a mechanical structure adapted to the farm environment and to the characteristics of the fruits and vegetables.^2,3,4 The robot hand plays a key role in automatic harvesting for different targets, and therefore its design is particularly important.^5,6,7

Robot hand are been adopted in many types of conditions, such as medical,⁸ industrial,⁹military¹⁰ and agriculturalMata et al^11–15.developed a novel underactuated multi-fingered soft robotic hand for prosthetic application.¹⁶ For Power and Precision Grasp of a prototype hand prosthesis, Leobardoet al. developeda Low-Cost EMG-Controlled Anthropomorphic Robotic Hand,¹⁷ which is based on a Six Degree-of-Freedom Open Source Hand designed by Krausz et alin 2016.¹⁸ In industry, Abd et al. designed an electrochemical mechanical polishing end-effector for robotic polishing applications.¹⁹Spadafora et al.²⁰ designed and constructed a robot hand prototype for underwater applications. To harvest the vegetablesor fruits in the fields, some previous researches were conducted for different targets. Because of the payload problem to the robot hand system, many previous researches were focused on the lightweight vegetables such as strawberry,^21–23 tomato,^7,24 cherry,²⁵ citrus^26,27 and Green Asparagus.²⁸ In addition, Muet al²⁹ designed an integrated end-effector for picking kiwifruit by robot in 2019.Roshanianfardet al¹ designed a pumpkin harvester robotic end-effector in 2017.

In this study, a robot hand-eye harvesting method was preferred by farmers for the advantage of robot arm can work point to point, which can reduce the damage to the fruits that are growing in the field. The robot system is developed by using three subsystems. The first one is machine vision system, which is going to detect the position of the fruits. The second one is robot hand-arm, which is utilized to grab the fruits and move it to the objective position or container. The third subsystem is moving vehicle, which is used to carry the robot hand-arm and machine vision system. For the first subsystem, an USB camera was utilized to grab images in the field in real time.And a deep neural network (DNN) based method was utilized to find the position of the pumpkin fruits.The weight of a pumpkin fruit is normally around 3 kg or more. It is required a high payload robot arm to provide enough power to pick up the pumpkin fruits. This paper focused on the development of the robot hand based on parameters of the dimension of pumpkin fruits.

Structure and working principle of the robot hand

The robot arm used in this study is a commercial robot arm UR10 (Universal Robots, Denmark) as shown in Figure1. The specification of the robot arm is shown in Table 1. The payload and reach region are 10 kg and 1.3 m, respectively. In addition, the power supply and power consumption are 100~240 VAC and 350 W, respectively.The robot arm is used for agriculture vehicles when the vehicles have a DC-AC power inverter.

A three-dimensional model of a robot hand based on a linkage mechanism for harvesting pumpkins was constructed by Fusion360 software, as shown in Figure 1.

(a) Robot hand mounted on the robot arm                                  (b) Structure of the robot hand

Figure 1 The structure of robot arm and robot hand.

The robot hand is composed by a base, a crank that is an active link, a connecting rod which is a connecting link, a hemispherical end effector, a connecting frame and a finger component.

The base is fixed to the robot arm. The active link, connecting link and hemispherical end effector are connected by a rotating pair. The hemispherical end effector consists of three parts of the same shape and is held together by a connecting bracket. The finger part is coupled to the hemispherical end effector by a rotating pair, and the torsion spring is mounted on the rotating pair. The finger part rotates to the outside under the action of the spring force.

The upper-end of the end effector is stopped by a limiting slot. The finger is rotated to a predetermined position and then stopped when touched the limiting slot. This allows the finger component to rotate only inward, which can act as a buffer to prevent the finger component from contacting the ground or the surface of a pumpkin.

The hemispherical end effector radius is 15 cm, which is decided by the pumpkin dimension parameters as shown in Figure 2. From this figure, the weight of the fruits is around 3 kg; the diameter and height are less than 25 cm.

Figure 2 The weight and dimension of pumpkin fruits.

In this design, the robot hand for pumpkin harvesting needs to be mounted on a robotic arm that is mounted on a machine that can walk in a pumpkin field, such as a tractor, a field walking robot, etc., to achieve a complete pumpkin picking system. When the camera captures the pumpkin, the control system processes it, then the arm moves to the predetermined position, then the robot opens, slowly closes, and picks up the pumpkin.

Kinematic model of the robot hand

Determining the geometrical parameters and kinematics of the linkage mechanism is very important. The robot hand is simplified to a four links mechanism as shown in Figure 3. A complex vector method is used to solve the kinematic relationship between the angular displacement and the rod length of the four links.

Figure 3 Notation of the Kinematic model for the four links mechanism.

The kinematic model is represented by Eq. (1).

$l_1\cdot e^{i\cdot {\varphi }_1}+l_2\cdot e^{i\cdot {\varphi }_2}=l_4+l_3\cdot e^{i\cdot {\varphi }_3}$ , (1)

where Euler equation：

$e^{i\cdot \varphi }=\cos \varphi +i\cdot \sin \varphi$ (2)

Through the above two formulas, we can get the following formula.

$\{\begin{array}{l}{l}_1\cdot \cos {\varphi }_1+l_2\cdot \cos {\varphi }_2=l_4+l_3\cdot \cos {\varphi }_3\\ l_1\cdot \sin {\varphi }_1+l_2\cdot \sin {\varphi }_2=l_3\cdot \sin {\varphi }_3\end{array}$ (3)

If the four-link length $\left(l_1,l_2,l_3,l_4\right)$ and the initial position $({\phi }_1)$ of the active link are known, we can get the exact position of the driven link by Eq. (3).

Therefore, ${\phi }_2$ and ${\phi }_3$ can be calculated by Eq. (4).

$\{\begin{array}{l}\tan \left(\frac{{\varphi }_3}{2}\right)=\frac{B\pm \sqrt{A^2+B^2-C^2}}{A-C}\\ {\varphi }_2=\arctan \left(\frac{B+l_3\cdot \sin {\varphi }_3}{A+l_3\cdot \cos {\varphi }_3}\right)\end{array}$ (4)

where,

$\{\begin{array}{l}A=l_4-l_1\cdot \cos {\varphi }_1\\ B=-l_1\cdot \sin {\varphi }_1\\ C=\frac{A^2+B^2+l_3{}^2-l_2{}^2}{2\cdot l_3}\end{array}$

Optimal design of the dimension for the robot hand

The length of each link in Figure 3 is the objective design variables in this paper. The length of the active link is set to a unit length. The initial position and the end position of the active link ($l_1$ in Figure 3) are given as random values, which are 68 degrees and 108 degrees, respectively. The initial and end positions of the follower are a function of the design variables, which are ${\theta }_1\left(x\right)$ and${\theta }_2\left(x\right)$ .

The optimal design objective (Eq. (5)) is to calculate a suitable dimension so that the robot hand can grab and release successfully.

$X={\left[\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right]}^T={\left[\begin{array}{ccc}{l}_2& l_3& l_4\end{array}\right]}^T$ (5)

The end effector needs to have a large opening angle tograb the largest pumpkin during the process of grabbing the pumpkin. The objective function, Eq. (6), is calculate the minimum of the difference between the opening angle and the target value of the end effector, to a target value 65 degrees.The target value as shown in Figure 4 (a) was determined so that the pumpkin can drop only by the gravity at the fully opened pose.

(a) Determine the design target value                                         (b) Optimal results

Figure 4 The output of optimization process.

min$f\left(x\right)={\theta }_2\left(x\right)-{\theta }_1\left(x\right)-\frac{65}{180}\cdot \pi$ (6)

In the calculation step, the following constrains were considered:

1. All three independent variables are larger than zero,
2. The initial position transmission angle is between 85 degrees and 90 degrees,
3. The initial position of the follower link is between 80 degrees and 90 degrees,
4. The angle between two connected links is limited to between 0 degrees and 170 degrees.

The initial length of $l_2,l_3$ and $l_4$ in Figure 3 are noted as,

$X_1={\left[\begin{array}{ccc}1& 1& 1\end{array}\right]}^T$ .

The length of $l_1$ in Figure 3 is defined as a unit length.

The optimization program is shown in Appendix A-1. The optimization results of the decided length of $l_2,l_3$ and in Figure 3 are,

$X={\left[\begin{array}{ccc}0.78& 0.84& 1\end{array}\right]}^T$ .

The output of the optimization process is shown in Figure 4.To simplify the calculation, the curved model was simulated by straight lines as shown in Figure 4 (a). The thin lines in Figure 4 (b)are the middle calculation results. The bold lines are the final optimal result, it shows that the closed pose. It satisfies the design objectives.

Creating a simulation prototype model

A multibody dynamics simulation solution software-Automatic Dynamic Analysis of Mechanical System (ADAMS) (MSC Software Corporation, USA)-was utilized to simulate the performance and motion laws of decided mechanical systems in section 2. The robotic components are first constructed under the Fusion 360 (Autodesk, USA) software and assembled into a 3D solid model, and then imported into the ADAMS environment to complete the virtual prototype. The 3D model shown in the ADAMS environment is shown in Figure 5 at holding and releasing pose.

Figure 5 Working condition shown in 3D model using ADAMS.

Simulation results and discussion

In the simulation, parameters were set as shown in Table 2, the driving was applied to rotate the crank at an angular velocity of 10 degrees per second. The simulation time is set to 3s and step is 0.1s.

The movement and speed of the end of the finger are shown in Figure 6. It is shown that the speed of the end is faster than the active link; moreover the speed increases with the time. Therefore, in the application the speed of the active link can be controlled to satisfy the requirement of grabbing speed.

Figure 6 Angular velocity of end of the robot hand.

The torque received by the crank is shown in the Figure 7. The torque is normally less than 10 Nm when there is a peak at 20 Nm in a short time when pumpkin is 5 Kg. Anactuator (motor and reducer) of the robot hand should provide more than 20 Nm for the designed robot hand.

Figure 7 Angular velocity of end of the robot hand.

A robot hand was designed to pick up the pumpkin fruits in the outside field for autonomously harvesting of pumpkin. A kinematic model of the four links mechanism was created, and an optimization design program was made to calculate the dimension of the mechanism to match the requirement of the robot hand so that to grab and release the pumpkin fruits. In addition, a dynamics simulation was done by using ADAMS software, the simulation results show that the end of the designed robot hand can open almost as the same speed of the active link (Link 1). And the torque required for the active link to finish the job of grab a pumpkin with 5 kg is 20 Nm.

This study is supported by the funds from “Cross-ministerial strategic innovation promotion program”.

The pumpkin fruits weight and dimension were measured at the field of Hokkaido Agricultural Research Center.

The Authors declare that there was no conflict of interest.

1. Roshanianfard A, Noguchi N. Designing of pumpkin harvester robotic end–effector. The 3rd International Conference on Control, Automation and Robotics; 2017; Japan; 2017.
2. Ali Roshanianfard, Noboru Noguchi. Characterization of pumpkin for a harvesting robot. IFAC–PapersOnLine. 2018;51(17):23–30.
3. Bac CW, van Henten EJ, Hemming J, et al. Harvesting robots for high–value crops: state–of–the–art review and challenges ahead. Journal of field robotics. 2014;31(6):888–911.
4. Masateru Nagata, Kenji Hiyoshi, Qixin Cao, et al. Basic Study on Strawberry Harvesting Robot (Part II): Design and Development of Harvesting Mechanism. IFAC Proceedings Volumes. 2000;33(29):55–59.
5. Fu LS, Zhang FN, Yoshinori G, et al. Development and experiment of end–effector for kiwi fruit harvesting robot. Trans Chin Soc Agric Mach. 2015;46(3):1–8.
6. Yuanshen Zhao, Liang Gong, Chengliang Liu, et al. Dual–arm Robot Design and Testing for Harvesting Tomato in Greenhouse. IFAC–PapersOnLine. 2016;49(16):161–165.
7. Guohua Wang, Yabo Yu, Qingchun Feng. Design of End–effector for Tomato Robotic Harvesting. IFAC–PapersOnLine. 2016;49(16):190–193.
8. Yanhuan Huang, Chingyi Nam, Waiming Li, et al. A comparison of the rehabilitation effectiveness of neuromuscular electrical stimulation robotic hand training and pure robotic hand training after stroke: A randomized controlled trial. Biomedical Signal Processing and Control. 2020;56:101723.
9. Liang Jie, Jia Shu–Hui. The Study of Design Method for Robotic Drill End Effector. Procedia Engineering. 2017;174:206–210.
10. Savas Dilibal R, Murat Tabanli, Adnan Dikicioglu. Development of shape memory actuated ITU Robot Hand and its mine clearance compatibility. Journal of Materials Processing Technology. 2004;155–156:1390–1394.
11. Lingxin Bu, Guangrui Hu, Chengkun Chen, et al. Experimental and simulation analysis of optimum picking patterns for robotic apple harvesting. Scientia Horticulturae. 2020;261:108937.
12. Jun Li, Manoj Karkee, Qin Zhang, et al. Characterizing apple picking patterns for robotic harvesting. Computers and Electronics in Agriculture. 2016;127:633–640.
13. Andreas De Preter, Jan Anthonis, Josse De Baerdemaeker. Development of a Robot for Harvesting Strawberries. IFAC–PapersOnLine. 2018;51(17):14–19.
14. Qixin Cao, Masateru Nagata, Yoshinori Gejima, et al. Basic Study on Strawberry Harvesting Robot (Part I): — Algorithm for Locating and Feature Extracting of Strawberry Fruits. IFAC Proceedings Volumes. 2000;33(29):49–54.
15. Zhiguo Li, Fengli Miao, Zhibo Yang, et al. An anthropometric study for the anthropomorphic design of tomato–harvesting robots. Computers and Electronics in Agriculture. 2019;163:104881.
16. Mata Amritanandamayi Devi, Ganesha Udupa, Pramod Sreedharan. A novel underactuated multi–fingered soft robotic hand for prosthetic application. Robotics and Autonomous Systems. 2018;100:267–277.
17. Sánchez–Velasco LE, Arias–Montiel M, Guzmán–Ramírez E, et al. A Low–Cost EMG–Controlled Anthropomorphic Robotic Hand for Power and Precision Grasp. Biocybernetics and Biomedical Engineering. 2020;40(1):221–237.
18. Krausz N, Rorrer R, Weir R. Design and Fabrication of a Six Degree–of–Freedom Open Source Hand. Transactions on Neural Systems and Rehabilitation Engineering. 2015:1–1.
19. Mohammad AEK, Jie Hong, Danwei Wang, et al. Synergistic integrated design of an electrochemical mechanical polishing end–effector for robotic polishing applications. Robotics and Computer–Integrated Manufacturing. 2019;55(A):65–75.
20. Spadafora F, Muzzupappa M, Bruno F, et al. Design and Construction of a Robot Hand Prototype for Underwater Applications. IFAC–PapersOnLine. 2015;48(2):294–299.
21. Ya Xiong, Cheng Peng, Lars Grimstad, et al. Development and field evaluation of a strawberry harvesting robot with a cable–driven gripper. Computers and Electronics in Agriculture. 2019;157:392–402.
22. Hayashi S, Shigematsu K, yamamoto S, et al. Evaluation of a strawberry–harvesting robot in a field test. Biosystems Engineering. 2010;105(2):160–171.
23. Yamamoto S, Shigehiko H, Sadafumi S, et al. Development of Robotic Strawberry Harvester to Approach Target Fruit from Hanging Bench Side. IFAC Proceedings Volumes. 2010;43(26):95–100.
24. Xiao Ling, Yuanshen Zhao, Liang Gong, et al. Dual–arm cooperation and implementing for robotic harvesting tomato using binocular vision. Robotics and Autonomous Systems. 2019;114:134–143.
25. Amatya S, Karkee M, Gongal A, et al. Detection of cherry tree branches with full foliage in planar architecture for automated sweet–cherry harvesting. Biosystems Engineering. 2016;146:3–15.
26. Yi Wang, Yan Yang, Changhui Yang, et al. End–effector with a bite mode for harvesting citrus fruit in random stalk orientation environment. Computers and Electronics in Agriculture. 2019;157:454–470.
27. Mehta SS, MacKunis W, Burks TF. Nonlinear Robust Visual Servo Control for Robotic Citrus Harvesting. IFAC Proceedings Volumes. 2014;47(3):8110–8115.
28. Gerard Kennedy, Viorela Ila, Robert Mahony. A Perception Pipeline for Robotic Harvesting of Green Asparagus. IFAC–PapersOnLine. 2019;52(30):288–293.
29. Longtao Mu, Gongpei Cui, Yadong Liu, et al. Design and simulation of an integrated end–effector for picking kiwifruit by robot. Information Processing in Agriculture. 2019.