automation technology for tomorrow’s food production · automation technology for tomorrow’s...
TRANSCRIPT
Automation Technology for
Tomorrow’s Food Production
Satoshi Yamamoto
Visiting Faculty, CPAAS, WSU
Senior Researcher, BRAIN, NARO
AgRA Webinar: October 29th 2014
1
2
Motivation for the automation
Sustainable
Reliability & Safety
50,000
60,000
70,000
80,000
90,000
100,000
110,000
120,000
130,000
140,000
1920 1940 1960 1980 2000 2020 2040 2060
Po
pu
lati
on
in
Jap
an
Year
*1920 – 2010: Statistics Bureau, Japan
*2010 – 2060: National Institute of Population and
Social Security Research, Japan
Peak: 2008
Food production
Efficient work
Information management
Automation Technology
How to keep
the current
level?
TOPICS
3
1. Back ground
2. Components of plant factory for strawberries in
BRAIN, NARO
3. 3D modeling of apple fruit in CPAAS, WSU
National Agriculture and Food Research Organization
4http://www.naro.affrc.go.jp/english/index.html
Researcher: 1,542 (April, 2013)
The fiscal 2013budget: 529M US$
(1US$ = ¥109)
Research institute under MAFF
Largest research
organization addressing
“agriculture, food and rural
communities”
5
Fruit
Planted
Area (2012)
Production
Quantity (2012)
Wholesale
Value (2011) a)
(ha) (t) (106 USD)
Tomato 12,000 722,400 1,522
Strawberry 5,720 163,200 1,573
Cucumber 11,600 586,600 1,444
Egg plant 9,860 327,400 805
Sweet Peppers 3,420 145,000 602
“Unshu”, Mandarins 43,700 895,900 1,496
Apple 37,400 793,800 1,199
MAFF
a) Calculated as 1 USD = 100 JPY.
0
5000
10000
15000
20000
1970
1975
1980
1985
1990
1995
2000
2005
2010A
rea H
arv
este
d (
ha)
California
Japan
Outline of strawberry production in Japan
6
Annual working hours (h/0.1ha) 2,000
Harvest season (months) 6 (December to May)
Average of planted area per producer (ha) 0.3
Planting density (plants/0.1ha) 7,000 – 8,000
Production (t/0.1ha) 3 – 5
* MAFF, 2007
Seedling
10%
Planting
4%
Fertilization
3%
Pest control
4%Cultivation
management
28%
Harvesting
23%
Sorting,
Packing
27%
Labor
management
1%
Percentage of working hours
Plant Factory for Strawberry Production
7
1. Movable bench
system2. Stationary
harvesting robot
3. Sorting &
packing robot
8
Movable bench system Space saving
Automated spraying
Saving energy cost
Increasing yield per area
Improvement labor condition
Movie
9
Measurement growth information
Movable Bench System Kinect
Growth information of
all plants every day
10
Depth ImageColor Image
Measurement growth information
Easy to extract
leaf area using
depth info
11
4 m
Feb. 23
Mar. 29
Apr. 26
Color
Depth
Color
Depth
Color
Depth
42 bedsMeasurement growth information
12
Lack of Iron
High EC or Water stress
Health diagnosis
Measurement growth information
Basic info of
plants: height &
width
13
Strawberry harvesting robot
<Development Target>
1. More than 60% success rate
2. 10s to pick & place a fruit
3. 0.1ha / night (8-12h)
4. No bruise
Prototype 1
• Basic type (no storing function)
• Cylindrical manipulator (3 DOF)
• Three camera
• Four halogen lamp
• Finger for cutting & holding stem
• Suction tube to cancel the depth
error
14
Prototype 2
• Five LED
• Through type photo sensor
• Tilting motion of robot hand
• logistic function for fruit
containers
Cylindrical
manipulator
15
Picking motion
Suction tube
Finger
Through type
photo sensor
a) Approach to a fruit with
suction tube
b) Move finger forward
c) Move finger & tube backward
16
Tiling motion before picking
a) Right direction b) Left direction
Two independent
air cylinders
18
Prototype 4 Movie• One LED
• Diffused photo sensor
• Bending motion for placementShibuya Seiki Co., Ltd.
19
Stationary type with movable bench Movie
Commercialized by Shibuya Seiki Co., Ltd.
20
Change of robot’s faces
• Simplicity
• Compactness
21
1. Binarization 3. Maturity assessment
2. Occlusion assessment
Image processing
4. Stem detection
22
y = 0.991x - 2.7616
R² = 0.9557
0
20
40
60
80
100
0 20 40 60 80 100
Est
imat
ion (
%)
Human eye (%)
y = 1.0633x - 1.3707
R² = 0.8207
0
20
40
60
80
100
0 20 40 60 80 100
Est
imat
ion (
%)
Human eye (%)
Amaotome Beni-hoppe
Difference of coloring among varieties
23
Fruit condition
0
10
20
30
40
50
60
70
80
90
100
Aisle
(Feb.)
Bed
(Feb.)
Fru
it c
on
dit
ion
(%
)
‘Beni hoppe’ cultivar
Aisle
(May)
Bed
(May)
A B C
D E
Bed SideAisle Side
A B C D E
24
Harvesting from bed side
Prototype 2
Prototype 3
Mayekawa mfg. Co., Ltd.
Waseda University
25
Harvesting from bed side
Hand-eye-camera
for stem detection
Stereo vision for
position detection
Movie
26
Reduction of influence of fruit condition Movie
Separate from
adjoining fruitsApproach Pick Place
27
Reduction of influence of fruit condition
End-effector
a) Vacuum b) Grip
Mobile bench Unit 7 DOF Manipulator
Coloration
Measurement Unit
Position
Detection Unit
Movie
28
Reduction of influence of fruit condition Movie
3DOF
Manipulator
Mobile
Bench Unit
7DOF
Manipulator
Vacuum Hand
Picking Hand
Coloration
Measurement Unit
Position
Detection Unit
29
Mini-summary for harvesting robot2003
2013
Stationary harvesting robot
Harvesting success rate: 40 – 70 %
2010
2006
• Cylindrical manipulator (3 DOF)
• Three camera, Four halogen lamp
• Finger for cutting & holding stem
• Suction tube
• Machine vision & software: Maturity, Occlusion…
• Finger shape
• Tilt function of robot hand
• Diffused photo sensorDon’t move
expensive
robot!
30
Strawberry sorting & packing robot
From harvesting
box to shipping
tray
31
Single layer 1Double layer Small pack
Single layer2 Single layer 3Hart shape
Shipping types
32
Strawberry sorting & packing robot
Supply unit
Sorting &
Packing unit
Movie
Single-layer tray
Returnable tray
33
Supplying unit
Camera
Manipulator
(3 DOF) Suction hand
Harvesting
container
34
Sorting & packing unit
Collision
Safe
Camera
Manipulator
(4 DOF)
Suction
hand
Single-layer trayReturnable tray
35
Strawberry sorting & packing robot (2)Machine vision:
Kinect
Conveyer for
harvesting containers
Fruit conveyer
Conveyers for
shipping trays
Machine vision:
Color camera
End-effector
Manipulator
36
Start
Supply fruits and shipping tray
Detect the suction point of target fruit in
harvesting container
Pick up fruit, move to digital camera
Weight and orientation of the held fruit
Place on shipping tray
Stop
Continue?
Kinect
Digital camera
37
Segmentation of fruits in harvesting container
Segmentation of
fruits using color &
depth info
38
Fruit orientation
Size & Orientation
V of HSV
R – G image
Movie
Maximum error: 25.1˚
MEAN : 0.3˚
SD: 5.1˚
39
Strawberry packing robot in grading line Movie
Packing Robot
IR sensor
Weight scale
Yanmar Green System Co., Ltd.
Color camera
40
Strawberry packing robot (Basic)
41
Mini-summary for sorting & packing robot
2013
2011
2007
Robot handSupplying unit
Packing robot
(Basic)Sorting & Packing robot
using KinectPacking robot in grading line
7 s / fruit
4 s / fruit
1.5 s / fruit < 1 s / fruit
More
than
human
ability!
42
3D modeling of apple fruit in CPAAS, WSU Density: important factor for evaluation of a fruit inner
quality.
Volume: not a common technique in a fruit sorting system.
Appearance: check using surface color information.
3D reconstruction for
fruit sorting system
using Kinect
43
Measurement setting
Kinect
Apple
LED
44
3D models using Kinect
How should we
use them?
CAD data can be download from website of GrabCAD.
Automated grading system
• Inner quality: density
• Appearance assessment
45
Summery
Packing robot
in grading line
Grading based
on 3D model
Stationary harvesting robot
Growth measurement
Movable bench
system
46
For the tomorrow’s food production… Is it time to get out of the plant factory?
Construction the preferred environment for automation will be important.
Consumer 3D sensor has changed the accessibility to 3D info..
Stem detection will be a key
for a robotic harvester…
Over the Row Sensor Platform (left),
Detection of apple fruits (right)
CPAAS, WSU (Prof. Karkee)
Simple hardware & smart software
47
Thank you for your attention!