Fruit Grading Robotfor Apple, Pear, Peach, and Tomato

Robotization in Agriculture

Field management

Residue process InformationShipment

Seedling productionPlant management

Harvest

Pre-Processing

Grading

Cutting sticking robotGrafting robot

Transplanting robot

Strawberry harvesting robot

Fruit grading robot

Leek pre-processing robot

Robotic sprayer

Autonomous tractorAutonomous rice transplanter

Autonomous combine harvester

Under developmentCommercialized

Soil sensor

PearApplePeachPersimmonTomatoOrange

Fruit Grading Robotsize, color, shape, disease, insect injurysugar and acid contents, residual chemical

②

Pusher

TV camera

Halfway stage

x

Lifter

①③④⑤

⑥

⑦

⑧

z

Blower

Fruit providing robot

Gradingrobot

Pusher

Rotation

Product Reception

Container Transfer

Bar-code reading

Fruit Grading by Robot

Upper Image Acquisition

Fruit Providing by Robot

Inside Quality Inspection

Fruit Packing

Labeling of Grading and Size

Box Closure and Sealing

Box Transfer

Shipping

Fruit providing robot and Grading robot

(Two 3 Degrees of Freedom Cartesian Coordinate Manipulators)

2250

2500

2800

(mm)

Stroke ： 1165mm

①④

③

⑤

②

100

467

308

290

Path

Speed

① ④③ ⑤② ④

1000mm/s

250 317 308 145 145 (mm)

(s)0.5 0.317 0.683 0.6 0.3 0.3

2.7 s

Cycle time of the stroke ： 4.25 s　　　　　　 Items

2.7 s （ Go ）0.4 s （ Initial movement ）

1.0 s （ Return ）0.15 s （ wait ）

回転

Performance of grading robot on a condition of 80% 24 fruit containers and 20% 15fruit containers:12/4.25×3600×0.8=8131 fruits10/4.25×3600×0.2=1694 fruits 　　Total 9825fruits/hour/robot

Rotation

Action and performance of fruit grading robot

(mm)

PC-A

PC-B

PC-C

PC-D

PC-E

PC-J

Trigger

Trigger

Trigger

Trigger

Trigger

Robot Controller

PhotoInterrupter

Bottom and side images

Top images

Image acquisition system

80

85

Suction pad Fruit conveying tray

Suction pressure at Blower　　　 Peach ： 30 kPa　　　 Pear ： 45 kPa　　　 Apple ： 45 kPa

147

Data carrier

85

(256byte EE ROM)

illuminationTV cameraCapture boardSoftware

Image processing

Direct lighting Indirect lighting

MirrorDiffuserLight

True color

HalationWhitish color

Camera

Light

Camera

Reflecting plate

Direct Lighting(DL)

PL filter Heat absorption filters

Fan

“DL” is an Ishii-original illumination equipment for image acquisition to make direct lighting possible.

Small power （ 50W ） and high conversion efficiency from electricity into light.

PL filter: makes no halation on inspecting part of work in image.

Heat absorption filter: absorbs heat to protect PL filter whose polarizing material is melted at 60 .℃

(more than 1000nm)

(800 ～ 1000nm)

DL (Direct Light) made by Ishii

Halogen lamp

Color TV Camera“8300BT3” (separated camera head, CIS Corporation) was used for image acquisition. 6 mm lenses with C mount and PL filters were used for no halation.

Specification

● Single CCD type image sensor

●Flame shutter type

(non-interlace)

●RGB original color filters

●works under 70 temp.℃●Random trigger shutter

● Variable shutter speed (1/1000s)

●Small, simple, light, and cheap

Specification

● Single CCD type image sensor

●Flame shutter type

(non-interlace)

●RGB original color filters

●works under 70 temp.℃●Random trigger shutter

● Variable shutter speed (1/1000s)

●Small, simple, light, and cheap

Extracted features from processed image

Items Processing

① SizeSize is measured from the maximum length or areaon upper image or calculated volume from severalimages.

② Color

Color value and degree of color distribution areMeasured based on R, G, and B color component ratio.

③ ShapeShape is measured as boundary-based features, region-based features, mathematical morphology, and so on.

④ BruiseBruise is detected by combination of hue, value,and chroma.

+Internal quality by photo-sensor

Information of fruit appearance

Side(1) Side(2)

Side(3) Side(4)

TopBottom

Side(1) Side(2)

Side(3) Side(4)

Top Bottom

Side(1) Side(2)

Side(3) Side(4)

Top Bottom

＋　 Internal quality information (Sugar & acid contents, rot, ….etc.)

Original images 　　 Color conversion images 　 Processed images

Nansui Criteria for fruit appearance grade Sep.24, 2002Color Ｌ１ Ｌ２ Ｌ３ Ｌ４ Ｌ５Hue(Whole) 135-175 175 130-180 0-190Hue(Top) 135 135 130Hue(Bottom) 175 175 180 190Reddish area(Bottom) 58 63 68 80Saturation(Whole) 268 270 272 280Shape Ｌ１ Ｌ２ Ｌ３ Ｌ４ Ｌ５Roundness(Top & Bot.) 110 115 120Complexity(Top & Bot.) 120 130 157Deformativity(Side) 130 180 320Bruise, Disease Ｌ１ Ｌ２ Ｌ３ Ｌ４ Ｌ５Serious injury(Top) 4 20 50Serious injury(Bottom) 6 30 70

Medium injury(Top) 12 25 100Medium injury(Bottom) 15 30 130

Slight injury(Top) 15 40 110Slight injury(Bottom) 30 60 160

Shipping quantity of fruits

0

500

1,000

1,500

2,000

2,500

1 6 11 16 21 26 31 36 41Producer Ｎｏ．

Fruit Ｎｏ．

L1

L2

L3

L4

L5

L2

Grading result of pear on Sep.24, 2002

1. Objective and fair judgment on grading operation c.f. Human judgment depends on time, place, operator’s physical condition….

2. Labor substitution makes intelligent farming instead of grading and packing operations. >>credit and trust of production district　3. Accumulation of product information　 makes traceability system and precision farming guidance. (The information have been kept only in operator’s memory so far.)

Effectiveness of Fruit Grading Robot Introduction(Technology innovation)

FertilizingChemical sprayingIrrigation……

Accumulation to DB

•Appearance•Internal quality

Data from grading system can be used for precision farming

Geographical data Grading system

Product

Farming support system based on GIS

Field information

Geographical features

環境

Weather information

Chemical information

Yield information

Quality information

Gradient

Mapping on GIS data

Soil analysis centerSoil analysis centerProduct information centerProduct information center

Agricultural Cooperative Association

Soil sensor

Grading robot

Intelligent farmingPrecision farming

Consumer Market (Distribution)

Product informationField information

Farming guidanceDSS for farmers

ResidueCarbonization

BRAND BRANDVoice of consumer Quantity and

marketing value

Analysis of soil and chemicalsGIS, DSS

Variable distribution channelMarketing route

BiomassRe-uses

Transport

Operation recordsSensing information

ID tags

FreshProduct

New flow of product and information

Information oriented field Information added product