Humans and Robots Team Up for a High-tech Fruit Harvest

With the bumper crop of apples in 2013, many Washington tree fruit growers dream of a day when automated technology helps bring in the harvest. Manoj Karkee, assistant professor with the Center for Precision and Automated Agricultural Systems at Washington State University, believes that day will soon be here.

Hand sensor holding an appleKarkee and his team of WSU scientists recently won a $548,000 U.S. Department of Agriculture grant to develop tree fruit harvesting technology where robots and humans work side by side.

“Due to the complexity of fruit identification in an orchard setting, collaboration between human and machine is very important. This is what’s unique,” Karkee said. “When the robot can’t deliver, humans will step in and vice versa.”

Growers keen on technology

The cost of seasonal labor is increasing and the availability of a semi-skilled labor force continues to become more uncertain. But will growers embrace robotic fruit harvesting?

“Growers are very, very interested in this technology and enthusiastically waiting for it,” Karkee said. “In three to five years we hope to have a prototype to demo in the field, and in another five years be able to point to where growers can adopt the technology.”

When apples are in clusters or obscured by leaves and branches, a robot requires complex algorithms and long computational time to identify them. Humans, on the other hand, can very quickly identify fruits in these situations.

Conceptual human–machine interface for collaborative fruit harvest. Any apples missed by computer identification are selected by the human partner who is viewing the apple tree image on a computer screen. Touching the apples onscreen to identify them tells the robotic arm where to find them for harvesting.

When the two work together in a mobile system in the field, the fruit is identified in real time faster than by human or machine alone.

Karkee will develop specialized robotic methods to harvest fruit with consideration for things like the delicacy of the fruit and the dynamics of picking fruit by hand.

Materials, method mimic human hand

To develop a prototype, Karkee and his team, which includes Karen Lewis, Changki Mo, and Qin Zhang, will determine how best to detach fruit from the tree. Pull? Rotate? Combined twist and pull?

The researchers will study growth patterns of various types of apples. They will record and analyze videos of hand motions taken during manual picking as well as analyze force and pressure data recorded by sensors placed on the hand.

This knowledge will be transferred to a robotic hand for a highly efficient fruit removal system.

A complimentary project directed by Karkee will identify materials that best mimic the human hand in order to create a robotic hand that won’t damage fruit.

Funding for the research was awarded through the National Robotics Initiative, a joint program of the National Science Foundation, USDA National Institute for Food and Agriculture, National Institutes of Health and National Aeronautics and Space Administration.

– by Sylvia Kantor