robotics and automation in the food industry || robotics and automation in the fresh produce...

© Woodhead Publishing Limited, 2013

16

Robotics and automation in the fresh produce industry N. Kondo , Kyoto University, Japan

DOI: 10.1533/9780857095763.2.385

Abstract : The first automated grading facilities for fruit and vegetables became available more than 10 years ago. Recently, machine vision and near infrared (NIR) technologies as well as mechatronics and computer technologies have been employed to make these facilities more sophisticated and have led to their use for many kinds of agricultural products. Robot technology has proved able to handle agricultural products delicately and with a high degree of precision, and to gather information to create a database of products every season. This information is then utilized as traceability data for consumers and as farming guidance for producers.

Key words : fruit, grading, precision agriculture, robot, automation, information, traceability.

16.1 Introduction It is no exaggeration to say that of all agricultural operations, those carried out post-harvest have long employed the most automated equipment, following a great deal of research (Miller and Delwiche, 1991 ; Okamura et al ., 1991; Rehkugler and Throop, 1986 ; Shaw, 1990 ; Tao et al ., 1990 ; Lu and Ariana, 2002 ), because the correct environments for the introduction of automated instruments, such as machine vision, sensing systems, robots and PCs, are already in place. Since about ten years ago, packing robots and palletizing robots have been a frequent feature in fruit grading facilities (Njoroge et al ., 2002 ), while grading robots (Kondo, 2003 ), which collect round-shaped fruits and inspect them using a machine vision sys-tem, are now being introduced in some East Asian countries. Mechanical systems

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3


386 Robotics and automation in the food industry

are more easily controlled within a facility than in the field, and harvested agri-cultural products are similarly more suited to automated processes than products on trees; as a result, automated systems are most frequently introduced within agricultural facilities. Automatic machines and robots have already removed the need for human operators to carry out heavy, dangerous and monotonous opera-tions, have enhanced the market value of products, led to increased uniformity, improved hygienic/aseptic production conditions, and finally, have given future generations the possibility to achieve economic sustainability in small high value farming operations (Kondo and Ting, 1988).

There has been worldwide concern about the quality of food supply because of recent problems with fruits, vegetables, and other processed foods: the problems include food poisoning by bacteria, illegal unregistered agricultural chemicals, lack of product authenticity, and so on. The impetus behind these trends may be attributed to an increase in consumer concern about health and well-being and a feeling that producers need to respond to these concerns by consistently provid-ing products of guaranteed quality. Agricultural products present specific chal-lenges in terms of quality inspection techniques that are not encountered with other industrial products: non-standard products must be inspected according to their appearance and internal quality, and in order for the product to be acceptable to consumers, only non-destructive methods can be used. Several sensors have been developed and used in assessing features relating to internal quality, includ-ing sugar content, acidity, rind puffing, rotten core, and other internal defects (Kawano, 2003 ; Ogawa et al ., 2005 ). Automatic systems and robots used in agri-culture then play another important role, as they are able to keep a precise record of their operations in databases. They then utilize that information for the next operation or store the data either for future use by the producer in decision-making or to provide traceability information for quality assured foods.

This chapter describes the use of automated and robotized systems in post-harvesting technologies of fruits and vegetables such as citrus fruits, deciduous fruits, and vegetables. Products are rarely the same color, size, or shape, with differences occurring even between examples of the same cultivar. Sensing sys-tems for products with diverse and unpredictable characteristics are a particularly important type of automated technology in this respect. In this chapter, machine vision, one of the fundamental components of an automatic grading system, is explained first in general terms and then with specific reference to four automated post-harvesting systems for leeks, citrus fruits, eggplants, and deciduous fruits. Finally, a traceability system for food safety and security is described based on the technologies of data collection and utilization of grading facilities.

16.2 Machine vision system as a key technology Machine vision is an essential technology for inspecting agricultural products. Many machine vision systems have replaced the human eye in agricultural opera-tions. Generally speaking, setting up a machine vision system requires the correct

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3

Robotics and automation in the fresh produce industry 387


lighting, hardware for image acquisition (usually a lens, TV camera, image cap-ture board, PC, and cables between camera and capture board) and software for image acquisition and processing. Although many post-harvest systems may operate in controlled environments, some systems may be exposed to harsh envi-ronments that are not suitable for unprotected mechanical and electrical compo-nents. At fruit grading facilities in particular, precautions must be taken against fruit trichome and bloom of peaches and pears, which sometimes accumulate to depths of several millimeters during the course of a single day, in order to pro-tect TV cameras and PCs. During summer, TV cameras must be able to operate in temperatures of up to 40°C in the facilities. Since agricultural products, such as fruit and vegetables, are extremely varied in nature, machine vision systems for post-harvest operations have also been developed to deal with more varied products.

Many fruits and vegetables have glossy surfaces because of their cuticu-lar layer. This may sometimes prove a barrier to the acquisition of high quality images of these products. The cuticle is a transparent layer impermeable to water and made from cutin and wax, which makes it glossy. Leaves, stems, fruits, and seeds have these layers, which play an important role in preventing water loss and evaporation through the surface. It has been observed that irregular fruit shapes can cause unexpected glares at certain points, or uneven illumination on the fruit surface, even when lighting conditions are perfectly adjusted for the fruit vari-ety. The number of glares sometimes exceeds the number of light sources due to the irregular shape of the fruit. Eliminating glare and reflections from the sur-roundings, along with achieving uniform illumination conditions for the fruit, are the most important goals in the construction of a machine vision system. Kondo ( 2006 ) demonstrated a method that uses a polarizing (PL) filter in front of a halo-gen lamp, while protecting the PL filter from heat.

Color cameras are the most common sensor in machine vision systems. Agricultural products exist in a wide variety of colors, which can provide infor-mation about their properties and condition. Color cameras create red, green, and blue (R, G, and B) component images and can convey colors similar to those perceived by human senses. Many cameras have been used in grading systems to measure the size, color, shape, and defects of fruits and vegetables and to look for evidence of germination in cell trays. Since the absorption band of plant parts that contain chlorophyll is around 670 nm, the R component often shows whether or not the product is mature. Photosynthesis requires relatively more R and B components, so more light is reflected from the leaves from the G component than from the R and B components.

Images from a monochrome camera show the brightness of objects. The level of brightness intensity can indicate shine on the surface of fruits such as egg-plants through the use of glare, which should be removed in color images (Kondo et al ., 2007 ), while it can also measure product size and shape. Most monochrome charge coupled devices (CCD) are sensitive from the visible to the infrared region. Because the reflectance of agricultural products in the infrared region (700–1100 nm) is higher than that in the visible region (400–700 nm), it can be said that such

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



CCD cameras also offer advantages in distinguishing agricultural products from other objects or from the background.

Ultraviolet (UV) light is classified into three types according to wavelength: UV-C (100–280 nm), UV-B (280–315 nm), and UV-A (315–400 nm). It is claimed that UV-C light has disinfectant effects and reduces the amount of bacteria, viruses and mold. It is well known that UV-B light causes sunburn, and it is fur-ther hypothesized that it leads to the creation of vitamin D. UV-A is called black light and attracts insects, many of which have sensitivities in this range. In the UV-A range, some flower petals have high reflectivity in order to attract insects. Some agricultural products contain fluorescent materials and often fluoresce in the visible region due to excitation caused by the UV-A light. This often helps in the detection of defects in fruits, plants, and meats (Bodria et al ., 2002 ; Kim et al ., 2001 ). UV-A light sometimes excites fluorescent substances. It is well known that UV light at 365 nm causes damaged orange fruit skins to fluoresce (Uozumi et al ., 1987 ) and the fluorescent images are used for detecting damaged fruits (Slaughter et al ., 2008 ; Kondo et al ., 2008 ) as shown in Fig. 16.1 .

X-rays are electromagnetic waves with wavelengths of 10–0.01 nm corre-sponding to frequencies of between 30 PHz and 30 EHz. They are shorter than UV rays and can be an effective way of inspecting the internal qualities of agricul-tural products due to the ease with which they can be transmitted. Although X-ray cameras may be used, another common means of obtaining images in this range is the use of a scintillation screen and a CCD camera. X-ray images are different from the images captured at other ranges, as shown in Fig. 16.2 , because they pro-vide transmissive, rather than reflective, information.

The technology of computerized tomography (CT) has grown remarkably in recent years. CT is a non-destructive technique for capturing food images, which allows visualization of the internal structures or quality. To date, X-ray CT has been used for internal quality inspection: maturity of green tomatoes (Brecht et al. , 1991 ), defects in melons and watermelons (Tollner 1993 ), internal qual-ities of peaches (Barcelon et al. , 1997 , 1999 ), stones in apricots (Zwiggelaar et al ., 1997 ), measurement of water content of apples (Tollner et al. , 1992 ), and

Fig. 16.1 Rotten orange fruit images (right part is rotten): (a) black and white image; (b) fluorescent image.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



evaluation of textural characteristics in nectarines exhibiting woolly breakdown (Sonego et al. , 1995 ). Recently, a defect caused by an insect pest ( Noctuoidea ) in apples was investigated (Ogawa et al. , 2003a ) as shown in Fig. 16.3 .

Thus, machine vision technologies are used for inspecting agricultural products in a wide range of frequencies, from X-rays to infrared rays. Recently images using Terahertz waves – longer wavelength rays between far infrared and milliwaves – have started to be used for agricultural products in this region (Ogawa et al. , 2003b , 2004 ), while it has been known that chemicals, including vitamins, sugars, amino acids, pharmaceuticals, and pigments, have specific absorption bands (Kawase et al. , 2003 ; Walther et al. , 2003 ; Wallace et al. , 2004 ; Yamaguchi et al. , 2005 ; Fukunaga et al. , 2007 ). New findings in agricultural products or biological organs are expected, because as yet no studies have been carried out in this area.

16.3 Vegetable preprocessing and grading systems There are many kinds of fruits and vegetables. The grading operations and whole systems depend on their physical properties and producers’ strategies. Here, four

Fig. 16.2 Orange fruit images by color camera and X-ray camera: (a) black and white image; (b) transmissive image (top view); (c) transmissive image (side view).

Fig. 16.3 An apple damaged by a harmful insect: (a) CT image; (b) actual damaged apple.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



different types of systems for leeks, oranges, eggplants, apples, pears, and peaches are introduced.

16.3.1 Peeling and grading of leeks It is essential that leeks are peeled before grading or shipping. The peeling oper-ation is sometimes called a preprocessing operation. To properly peel the skin, the border plate between stalk and root must be cut. Because the plate is only a few millimeters thick, it is essential that the precise position of the plate is deter-mined correctly before cutting. If it is cut too close to the stalk, the inner stalks are exposed and the market value drops because the inner stalks dry out during trans-portation. If it is cut too close to the root, the skin cannot be easily peeled, because the outer skin adheres to the plate. For this reason, a machine vision system that enables the precise detection of the position of the border plate between stalk and root in leeks is a key technology for leek peeling. Figure 16.4 shows an image of the plate between stalk and root.

Figure 16.5 shows a leek preprocessing system currently in use in several Japanese agricultural facilities. Operators manually feed leeks onto the line of preprocessing machines, and once the position of the border plate is determined, the leeks are cut to a length of 60 cm. Before the position of the border plate posi-tion is determined, a blower is used to remove sandy soil attached to the lower stalk. In the second step, a three-fold root cutting operation is conducted because a single precise cut is not easy to achieve, due to the presence of soil and the irregular shape of the roots. In the last two cutting operations, two sets of mono-chrome cameras and color cameras are used. The monochrome cameras indicate the position of the border plate, while the color cameras detect the border between green leaves and white stalk for the third step. In the third step, the positions of the leeks on the line are adjusted so that the borders between green leaves and white stalks match with the positions of installed air nozzles. In the fourth step, high pressure air from a compressor is blown through nozzles and 20 leeks are peeled at a time.

Fig. 16.4 An image of a leek root just before cutting at the plate between stalk and roots.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



In a recently used grading system for leeks, the grading operation is combined with the skin peeling operation described above. After peeling, several cameras inspect the length, width and bend of the white stalk as well as the number of leaves. Defects are identified from the color and monochrome images as shown in Fig. 16.6 . Based on the results of the inspection process and on their weight, they are sorted into different grades. The most important evaluation indices are white stalk length and width, because the white part of the stalk is the edible part. A binding machine groups the leeks into bunches, and these are then packed into boxes for shipping.

16.3.2 Grading of oranges Orange grading operations have been automated for the last few decades. The system utilizes image-processing techniques and improved engineering design to convey fruits to the sensor, and then to detect fruit size, shape, color, maturity level, and taste. The system inspects each fruit with color CCD cameras stationed at six different angles to provide images of the fruit from all sides as shown in Fig. 16.7 . Lighting is provided through lighting devices using halogen lamps or light-emitting diodes (LED) fitted with PL filters.

Adjustmentline

Leaf cutting

To grading

Peeling Positionadjustment

Root cutting(three times)

Feeding

Fig. 16.5 Leek preprocessing system (SI Seiko Co. Ltd).

Fig. 16.6 Acquired images of leek by color and monochrome cameras (SI Seiko Co., Ltd): (a) whole color image; (b) monochrome leaf image; (c) monochrome lower stem

image.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



The labor-intensive process of hand-sorting fruit, traditionally considered the only accurate grading method, is completely eliminated by having a fully auto-mated conveyance system that ensures that oranges are fed to the sensor at a con-stant speed. Unmanned line inspection has helped to solve the problem of labor shortage caused by an aging population and an insufficient number of successors, which has been experienced in most production areas. Specially designed rollers rotate the fruit 180° ensuring that the cameras can capture a complete view. Two basic inspection stages can be identified in this system: external fruit inspection and internal fruit inspection.

In the external inspection stage, images from the CCD cameras set under random trigger mode are copied to the image grabber board fitted to the image-processing computer whenever a trigger occurs. The images are processed using specific algo-rithms for detecting color, size, bruises and shape. For internal fruit inspection, an NIR sensor determines the sugar content (brix equivalent) and acidity level of the fruits from light wavelengths received by spectrometers after light is transmitted through the fruit. The sensor photo-electrically converts the light into signals and sends them to the computer unit where they are processed and classified. In addi-tion, the internal fruit quality sensor measures the granulation level of the fruit, which indicates its internal water content. Rind puffing, a biological defect that occurs in oranges, is inspected using an X-ray sensor. Output signals from image processing are transmitted to the judgment computer where the final grading deci-sion is made based on features of the fruit and internal quality measurements.

The speed of the conveyor in a citrus fruit grading system is usually 60 m/min and several fruits are processed per second on each line. An important fea-ture of the system design is that it can be adapted for the inspection of many other products, such as potato, tomato, persimmon, waxed apple, and kiwi fruit, with the only modifications required being adjustments to the processing codes. Multiple lines for orange fruit inspection, combined with high speed conveyors and microprocessors allow the system to handle large batches of fruit in a very short time. All the information collected, from the point when the fruit arrives at

Judgement PC

Sugar and acid PC

Light projector X-ray generator Camera B in Camera B out Spin, 180�C turn

X-ray camera Camera A in Camera A Out Camera A top DL Light Camera B top

X-ray image PC Image A PC Image B PC

Light interceptor

Fig. 16.7 A schematic diagram of camera and lighting setup for citrus fruit grading. (Source: Njoroge et al ., 2002 .)

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



the collection site up to grading, packing, and shipping is stored in a database pro-viding farmers with product-specific guidance (Njoroge et al ., 2002 ).

16.3.3 Grading of eggplant Another type of grading system, designed for fruit with an elongated shape, such as eggplant, is also currently in use (Kondo et al ., 2007 ). This inspects not only size, shape, color, and defects but also the gloss on the fruit surface. The grading line consists of feeding, singulating, grading, and sorting conveyors as shown in Fig. 16.8 . Two sets of machine vision systems for grading fruit are installed in the grading conveyor, one before the fruit is rotated, the other after. Six color TV cameras provide images of the color, size, shape, and defects of the fruit (Chong et al ., 2008a ), while four monochrome cameras assign a degree of surface gloss to each fruit (Chong et al ., 2008b ) as shown in Figs. 16.9 and 16.10 respectively.

Recently, the ability to inspect all sides of each fruit has become a highly desirable feature of a grading system. A rotary tray was developed, for use at a certain point on a grading line, in order to rotate the fruit while the color and monochrome machine vision systems carry out the quality inspection, as shown in Fig. 16.8 . Each grading line consists of 348 rotary trays and runs at a speed of 38.1 m/min. Since this grading machine has six lines, it is capable of sorting a total of 504 000 fruits per day. This translates into an estimated potential processing capacity of 40.320 t per day.

16.3.4 Fruit grading robot A grading system using robots has been developed for use with deciduous fruits such as peaches, pears, and apples. This system automatically picks fruit from containers and inspects all sides of the fruit (Kondo, 2003 ). Figure 16.11 shows

Section offruit feeding NIR Camera

Color Camera A

Color Camera B

Color Camera E

Color Camera D

Turning over

Monochrome Camera B

Monochrome Camera D

Sorting Conveyor

Returning

Conveyor

Fruit Feeding

ConveyorSingulating

Conveyor

Color Camera C

Color Camera F

Grading ConveyorMonochrome Camera A

Monochrome Camera C

Inspection system by TV cameras

Inspection system by TV cameras

Fig. 16.8 Inspection line of an eggplant grading system. (Source: Kondo et al ., 2007 .)

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



the robot that picks the fruit from the containers (the providing robot) and the robot responsible for grading the fruit (the grading robot). A fruit container for 6 fruits × 5 lines (30 fruits), 6 fruits × 4 lines (24 fruits), or 5 fruits × 3 lines (15 fruits), depending on fruit size, is moved into the working area of the providing robot by a pusher (1). The providing robot has a three degrees-of-freedom (DOF) Cartesian coordinate manipulator and six suction pads as end-effectors. The suc-tion pads come down and collect six (or five) fruits (2), which are then transferred to a halfway stage (3). Two providing robots work independently and move 12 (or 10) fruits to this halfway stage. A grading robot which consists of another 3 DOF manipulator (two prismatic joints and a rotational joint) and 12 suction pads picks them up again (4) and moves them to trays on the conveyor line. Images of the underside of the fruits are acquired as the grading robot moves over 12 TV color cameras, while four side images of fruits are acquired by rotating the fruits by 270°. The cameras turn for 90° following the movement of the grading robot (5). Once the image has been acquired, the robot releases the fruits into trays (7) and a pusher moves the trays to conveyor line (8).

This grading robot’s maximum speed is 1 m/s and its stroke is about 1.2 m. It takes the robot 2.7 s to transfer 12 fruits to trays, 0.4 s to move down to the con-veyor line, and 1 s to return once fruits have been released. 0.15 s are spent wait-ing for the next batch of fruit The total time to complete the operation is 4.25 s. This means that one set of robots can process approximately 10 000 fruits per hour and four robot sets can process 100 t in a day, if a fruit is assumed to weigh 350 g. This robot can be used in the grading of many round-shaped fruits. Figure 16.12 shows top images of 11 varieties of deciduous fruits – peaches, pears, and apples – which were graded by the grading robot.

Fig. 16.9 Color images of eggplant: (a) original image; (b) binary image; (c) processed image (calyx, bruises and center line of fruit were detected).

Fig. 16.10 Monochrome images: (a) gloss surface fruit; (b) dull surface fruit.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



A NIR inspector installed on each line measures the sugar content and internal qualities of the fruit after images have been captured by the machine vision sys-tem. Based on the inspection results, fruit is sorted into several grades and several sizes. After inspecting both the internal and external quality, another robot with 12 three DOF Cartesian coordinate manipulators packs fruits into a cardboard box. An inkjet printer can print the grade and size on the surface of the box on the line, and the box is then sealed. The boxes are palletized by an articulated robot before storage.

Blower

Gradingrobot

Providingrobot

Halfway stage

TV camera

PusherCarrier

LifterPusher

z

x

4

3 21

5

6

7

8

Fig. 16.11 A grading robot for deciduous fruits. (Source: Kondo, 2003 .) Numbers in circles indicate: 1, pusher pushes 2 containers; 2, providing robot sucks 12 fruits together; 3, 12 fruits are spaced away from each other and are moved to halfway stage; 4, grading robot sucks 12 fruits together; 5, bottom images of 12 fruits are acquired by 12 TV cameras during moving from halfway stage; 6, 4 side images of 12 fruits are acquired during fruit rotation by 90 degrees turn down of the 12 TV cameras; 7, 12 fruits are released on carriers;

8, carriers are pushed to a grading line.

Fig. 16.12 Top views of apples, peaches and pears handled by the grading robot (upper images are apples, lower images are a peach and four pears).

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



16.4 Information flow for food traceability and farming guidance

As described in previous sections, in some grading facilities most operations are automated or robotized. That means that most of the information relating to grad-ing can be stored in a database and be linked with other data such as producer ID, field ID, operation records kept by producers, weather information, geographical information, and so on. If the other operations were also robotized and automated, many kinds of information could then be obtained by the robots or automated sys-tems. Operations at the production stage may be classified into several categories: field management, tree management, harvesting, and grading operations. Figure 16.13 shows the operations and possible information obtained on each operation at the production stage as well as at the distribution and consumption stages.

Figure 16.14 shows a flow of data collection from field management in the production stage up to the consumption stage, with particular emphasis on infor-mation gathered by the grading robot, when data from the robotic system were stored in a database. Containers from producers have individual barcodes, which provide the producer ID, field ID, fruit variety and number of fruits in the con-tainers to a PC. The data relating to each fruit processed by the grading robot are automatically stored in a PC and immediately sent to each Radio Frequency Identification (RFID) in the carrier through an antenna and a ROM-writer. Based on the data, each fruit in the carrier is sorted and packed into a box. In the box,

Field ID (address, height above sea level), producer IDClimate, weather information (temperature, humidity, irradiation, precipitation)

Transportation environmental condition (temperature, humidity)Package type and method, transporting method, time, and distance

Distribution information

Selling price, selling time, information of arrival of goods, quantity to sell,opinions of consumers (on taste, freshness, etc.)

Consumption information

Production information

Field management Seedling productionCrop management

Harvesting Grading

Operation record:Irrigation, fertilization,chemical spraytransplanting, pruning

3D location of productHarvesting time anddate

Appearance (color, size,shape, bruise, disease),internal quality (sugarand acid contents, rottencore, inside defect),graded rank, chemicalresidue, ingredients,district name

Crop ID, size, color,disease

Plant variety, crop ID,size, color, disease

N, P, K, pH, ECSOM, MC, soiltemperature,compactness.

Fig. 16.13 Information accumulated in a database.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



the fruit can still be matched with the fruit data stored thanks to the position of the fruit in the PC as long as this position is maintained. All the fruit data are collected in a traceability database, and are linked to the other production data shown in Fig. 16.13 . The other data may be inputted to the traceability database in another way. If a problem occurs, rendering it necessary to trace all the data up to the production stage, it is possible to check the data of any fruit in a box at the distri-bution or consumption stage by typing the product ID and the fruit position in the box into a PC. There is another useful application of this information: it is obvious that the relationship between farming operations and the grading data collected in the database every year can be used by farmers as a tool for achieving precision agriculture. Thus, the automated grading systems and robots can assist farmers through their accurate collection of many kinds of information. That means that the data accumulated in a database can be used for traceability to ensure food safety and security foods and for the provision of farming guidance to producers.

The most significant problem is then the question of who can manage such a large amount of data as that collected by the robots, sensors and producers. Figure 16.15 shows a solution whereby the management of the data is undertaken by an agricultural cooperative association (or company) or local farmers’ group. The agricultural association may establish a soil analysis center or product information center with inspection and analysis devices and sensors, as well as a database for conducting precision agriculture in the local area. The association can make use of many kinds of data for producers, because field information from a soil sensor or other methods, product information from the grading robot, and producers’

Grading robot

ROM-Writer

Sorting

IC memory

Product ID

Packingrobot

123ABC

Product IDconsumer

Barcodereader

Reception ID Producer ID Field ID Received date Variety Container number Fruit number

Grading ID Size Color Shape Defect

PC

Soil sensor

Field information

Producer

Tracebility database

http://www.***

Farming operationrecordsChemicalsFertilizers

Product ID issued date and timePacking robot No.Grading and reception information

Transportation dataEnvironmental condition

Sales dataPrice, quantity

Fig. 16.14 A flow of data from grading robot for traceability information.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



operation records are stored in the database. All the data are linked and analyzed on a geographical information system (GIS) and each producer is informed of the operations necessary for improved yields and higher quality products by a deci-sion support system (DSS).

From a different perspective, all the information can be made available to con-sumers and distributors through a web site. A trusted high quality product with precise traceability data may increase the market value as well as producing a local regional brand with a good reputation. If there is a problem with any of the products, quick action to resolve the issue is possible based on the opinions of consumers or distributors and on linked data in the database. This risk manage-ment is also an important role of the traceability system.

16.5 Conclusion In recent years, problems with the quality of food supply are becoming increas-ingly significant worldwide, with particular concerns surrounding, imported or exported foods due to the different criteria and unfamiliar food culture in many countries. To solve these problems, a food traceability system, which can show the history of the food during production, distribution and consumption, is desir-able. Bioproduction robots and automated systems have replaced human labor

Soil sensor

Information oriented fields

Field information

BiomassRe-uses

GIS, DSS

Voice of consumer

Consumer

Flow of product and information

Analysis of soil and chemicals

Operation recordsSensing information

Farming guidanceDSS for farmers

AgriculturalCooperative Association

Soil analysis centerProduct information center

Freshproduct

Productinformation

Information added product

Grading robot

Transport

Variable distribution channelMarketing route

Quantity andmarketing value

Market (Distribution)

Intelligent farmingPrecision farming

Residue

Carbonization

BRAND BRAND

Fig. 16.15 Data management by agricultural cooperative association.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



and have contributed to increasing the market value of products and producing uniform products. In addition, recent experiences show that these robots and auto-mated systems can also be useful for gathering and storing information on agricul-tural products and operations. Since the product information and operation history leads to foods with a higher level of safety and security the data can be useful to consumers in terms of food traceability. Furthermore, if information on post-harvesting operations is linked with that from field operations, the resulting data can provide farming guidance to producers and can offer a valuable contribution to the development of precision agriculture systems.

16.6 References Barcelon , E. G. , S. Tojo and K. Watanabe , 1997 . X-ray CT scanner for detecting inter-

nal changes in peach. Proceedings of the International Symposium on Agricultural Machinery and Automation , Taipei , Taiwan , 227–232.

Barcelon , E. G. , S. Tojo and K. Watanabe , 1999 . X-ray computed tomography for inter-nal quality evaluation of peaches , Journal of Agricultural Engineering Research , 73 , 323 –330.

Bodria , L. , M. , Fiala , R. Oberti and E. Naldi , 2002 . Chlorophyll fluorescence sensing for early detection of crop’s diseases symptoms, ASAE Paper Number 021114.

Brecht , J. K. , R. L. Shewfelt , J. C. Garner and E. W. Tollner , 1991 . Using X-ray-computed tomography to nondestructively determine maturity of green tomatoes . HortScience , 26 , 45 –47.

Chong , V. K. , N. Kondo , K. Ninomiya , T. Nishi , M. Monta , K. Namba and Q. Zhang , 2008a . Features extraction for eggplant fruit grading system using machine vision . Applied Engineering in Agriculture, ASABE , 24 (5), 675 –684.

Chong , V. K. , T. Nishi , N. Kondo , K. Ninomiya , M. Monta , K. Namba , Q. Zhang and H. Shimizu , 2008b . Surface gloss measurement on eggplant fruit . Applied Engineering in Agriculture , 24 (6), 877 –883.

Fukunaga , K. , Y. Ogawa , S. Hayashi and I. Hosako , 2007 . Terahertz spectroscopy for art conservation . IEICE Electronics Express , 4 , 258 –263.

KAWANO, S., 2003. Handbook for Food Non-destruction Measurement , Science Forum Inc. Kawase , K. , Y. Ogawa , Y. Watanabe and H. Inoue , 2003 . Non-destructive terahertz imag-

ing of illicit drugs using spectral fingerprints . Optics Express , 11 , 2549 –2554. Kim , M. S. , Y. R. Chen and P. M. Mehl , 2001 . Hyperspectral reflectance and fluorescence

imaging system for food quality and safety . Transactions of the ASAE , 44 (3), 721 –729. Kondo , N. , 2006 . Machine vision based on optical properties of biomaterials for fruit grad-

ing system . Environment Control in Biology , 44 (3), 3 –11. Kondo , N. P. , P. Ling , M. Kurita , P. D. Falzea , T. Nishizu , M. Kuramoto , Y. Ogawa and Y.

Minami , 2008 , A double image acquisition system with visible and UV LEDs for citrus fruits. Proceedings of Food Processing Automation Conference on CD-ROM, ASABE , June 28 2008, Rhode Island, USA.

Kondo , N. and K. C. Ting , Eds., 1998 . Robotics for Bioproduction Systems . ASAE , St. Joseph, MI, 325p.

Kondo , N. , 2003 . Fruit grading robot. In Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics on CD-ROM, July 20–24 2003, Kobe, Japan.

Kondo , N. , K. Ninomiya , J. Kamata , V. K. Chong , M. Monta and K. C. Ting , 2007 . Eggplant grading system including rotary tray assisted machine vision whole fruit inspection . Journal of the JSAM , 69 (1), 68 –77.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3



Lu , R. and D. Ariana , 2002 . A near-infrared sensing technique for measuring internal qual-ity of apple fruit . Applied Engineering in Agriculture , 18 , 585 –590.

Miller , B. K. and M. J. Delwiche , 1991 . Peach defect detection with machine vision . Transactions of ASAE , 34 (6), 2588 –2597.

Njoroge , J. , K. Ninomiya , N. Kondo and H. Toita , 2002 . Automated fruit grading system using image processing. In Proceedings of SICE Annual Conference 2002 , Osaka , Japan , MP18-3 (CD-ROM).

Ogawa , Y. , N. Kondo and S. Shibusawa , 2003a . Inside quality evaluation of fruit by x-ray image. In Proceedings of the 2003 IEEE/ ASME International Conference on Advanced Intelligent Mechatronics (AIM2003) on CD-ROM, 1360–1365.

Ogawa , Y. , K. Kawase , H. Tashiro and M. Yamashita , 2003b . Water content measurement in plants by THz-imaging, THz2003 Technical Digest: 93.

Ogawa , Y. , K. Kawase , M. Mizuno , M. Yamashita and C. Otani , 2004 . Nondestructive and real-time measurement of moisture in plant . IEEE Journal of Transactions on ELS , 124 (9), 1672 –1677 (in Japanese).

Ogawa , Y. , N. Kondo and S. Shibusawa , 2005 . Internal quality evaluation of fruit with X-ray CT . Journal of Society of High Technology in Agriculture , 17 (2), 75 –83.

Okamura , N. K. , M. J. Delwiche and J. F. Thompson , 1991. Raisin grading by machine vision. ASAE Paper No. 91–7011.

Rehkugler , G. E. and J. A. Throop , 1986 . Apple sorting with machine vision . Transactions of ASAE , 29 (5), 1388 –1397.

Shaw , W. E. , 1990 . Machine vision for detecting defects on fruits and vegetables. Food Processing Automation, Proceedings of the 1990 Conference , ASAE : 50–59.

Slaughter , D. C. , D. M. Obenland , J. F. Thompson , M. L. Arpaia and D. A. Margosan , 2008 . Non-destructive freeze damage detection in oranges using machine vision and ultraviolet fluorescence , Postharvest Biology and Technology , 48 , 341 –346.

Sonego , L. , R. Ben-Arie , J. Raynal and J. C. Pech , 1995 . Biochemical and physical eval-uation of textural characteristics of nectarines exhibiting woolly breakdown: NMR imaging, X-ray computed tomography and pectin composition . Postharvest Biology and Technology , 5 (3), 187 –198.

Tao , Y. , C. T. Morrow , P. H. Heinemann and J. H. Sommer , 1990 . Automated machine vision inspection of potatoes. ASAE Paper No. 90–3531.

Tollner , E. W. , 1993 . X-ray technology for detecting physical quality attributes in agricul-tural produce . Posthavest News and Information , 4 (6), 149N – 155N .

Tollner , E. W. , Y. C. Hung , B. L. Upchurch and E. E. Prussia , 1992 . Relating X-ray absorp-tion to density and water content in apples . Transactions of the ASAE , 35 , 1921 –1928.

Uozumi , J. , S. Kawano , M. Iwamoto and K. Nishinari , 1987 . Spectrophotometric sys-tem for the quality evaluation of unevenly colored food . Journal of the JSFST , 34 (3), 163 –170.

Wallace , V. P. , P. F. Taday , A. J. Fitzgerald , R. M. Woodward , J. Cluff , R. J. Pye and D. D. Arnone , 2004 . Terahertz pulsed imaging and spectroscopy for biomedical and pharmaceutical applications . Faraday Discussions , 126 , 255 –263.

Walther , M. , B. M. Fischer and P. Uhd Jepsen , 2003 . Noncovalent intermolecular forces in polycrystalline and amorphous saccharides in the far infrared . Chemical Physics , 288 , 261 –268.

Yamaguchi , M. , F. Miyamaru , K. Yamamoto , M. Tani and M. Hangyo , 2005 . Terahertz absorption spectra of L-, D-, and DL-alanine and their application to determination of enantiometric composition . Applied Physics Letters , 86 , 053903 .

Zwiggelaar , R. , C. R. Bull , M. J. Mooney and S. Czarnes , 1997 . The detection of soft materials by selective energy X-ray transmission imaging and computer tomography . Journal of Agricultural Engineering Research , 66 (3), 203 –212.

Cop

yrig

hted

Mat

eria

l dow

nloa

ded

from

Woo

dhea

d Pu

blis

hing

Onl

ine

D

eliv

ered

by

http

://w

ww

.woo

dhea

dpub

lishi

ngon

line.

com

M

onas

h U

nive

rsity

(76

5-47

-440

)

Tue

sday

, Mar

ch 1

2, 2

013

7:14

:08

PM

IP A

ddre

ss: 1

30.1

94.2

0.17

3

robotics and automation in the food industry || robotics and automation in the fresh produce...

Documents