cakmak cse571 march 2019 - university of washington...human-robot interaction taxonomy h. yanko and...
TRANSCRIPT
![Page 1: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/1.jpg)
SPECIAL TOPICS: HUMAN-ROBOT INTERACTION
Maya Cakmak
![Page 2: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/2.jpg)
HUMAN-ROBOT INTERACTION
Illustrations by Josh Ellingson
commanding a robot collaborating with a robot
GOAL: More effective and intuitive interactions
![Page 3: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/3.jpg)
HUMAN-ROBOT INTERACTION
Illustration by Jorge Cham
programming a robot
GOAL: More effective and intuitive interactions
![Page 4: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/4.jpg)
HRI -VS- ROBOTICS
ROBOT
actions
statesENVIRONMENTtask
ROBOT
actions
statesHUMAN
+ human state + explicit human input
+ robot communicative actions
task
![Page 5: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/5.jpg)
HRI -VS- HCI
user system
INTERFACEsystem
a lifetime of
interactions
inputs
outputs
![Page 6: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/6.jpg)
ANTHROPOMORPHISM
• the tendency to attribute human characteristics to
inanimate objects, animals and others
![Page 7: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/7.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
![Page 8: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/8.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
๏ task: urban search&rescue, walking aid for the blind, toy, delivery robot
๏ criticality: high, medium low
![Page 9: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/9.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
![Page 10: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/10.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
๏ # of humans/# of robots
๏ homogeneous, heterogeneous
![Page 11: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/11.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
Figure 1. The possible combinations of single or multiple humans and robots, acting as individuals or in teams.
H.G.F.E.
D.H H
R
H H
R
H H
R R
H H
R R
B.A.H
R
H
R R
H
R R
C.
R R
H H
![Page 12: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/12.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
๏ supervisory, operator, teammate, mechanic/programmer, bystander
![Page 13: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/13.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS, PROVIDED-SENSORS, SENSOR-FUSION, PRE-PROCESSING
• TIME, SPACE
• AUTONOMY, INTERVENTION
๏ avoiding, passing, following, approaching, touching, none (not co-located)
๏ synchronous, asynchronous
๏ co-located, non-co-located
![Page 14: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/14.jpg)
HUMAN-ROBOT INTERACTION TAXONOMY
H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International Conference on Systems, Man and Cybernetics, 2004.
• TASK, CRITICALITY
• ROBOT-MORPHOLOGY
• HUMAN-ROBOT-RATIO
• ROBOT-TEAM-COMPOSITION
• SHARED-INTERACTION-LEVEL
• INTERACTION-ROLES
• PHYSICAL-PROXIMITY
• AVAILABLE-SENSORS…
• TIME, SPACE
• AUTONOMY, INTERVENTION
๏ adjustable autonomy
๏ sliding-scale autonomy
๏ mixed-initiative
๏ supervised autonomy
๏ symbiotic autonomy
![Page 15: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/15.jpg)
TOPIC 1
ROBOTS LEARNING FROM HUMANS
![Page 16: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/16.jpg)
VISION
End-user programmable general-purpose robots
![Page 17: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/17.jpg)
VISION
BAXTER, RETHINK
GP8 PALLET, SEEGRID
“Baxter can be taught via a GUI and through direct manipulation of its robot arms. That means non-technical, hourly workers can train and retrain it right on the line.”
“it is very easy to train the robot by simply first walking it through the route it is to take, load an item it is designed to transport and then push the ‘go to work’ button”
End-user programmable general-purpose robots
![Page 18: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/18.jpg)
WHY IS IT IMPORTANT?
Because we cannot predict...
‣how they want it done
‣variability in conditions
‣what users want
![Page 19: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/19.jpg)
WHY IS IT CHALLENGING?
Existing tools assume good teachers...
‣ large number of demos
‣variance in demos
‣smooth/consistent demos
... everyday users are not!
‣ inaccurate mental model
‣ limited time, patience, attention, memory
CHALLENGE: BETTER DEMONSTRATIONS, FASTER!
![Page 20: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/20.jpg)
ACTIVE LEARNING
CATEGORY 1 CATEGORY 2
? ?
?
QUERIES
![Page 21: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/21.jpg)
ACTION 1 ACTION 2
STATES
?
What do I do in this state?
ACTIVE LEARNING IN ROBOTICS
Oudeyer 2007, Grollman 2007, Robbel 2007, Chernova 2009, Rosenthal 2009, Kroemer 2009, Gribovskaya 2010, among others.
![Page 22: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/22.jpg)
MANIPULATION SKILLS
![Page 23: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/23.jpg)
CONTINUOUS ACTION SPACES
∈
�(s) = a
STATErelative 6D end-effector
configuration
SKILL POLICY
![Page 24: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/24.jpg)
CONTINUOUS ACTION SPACES
∈
�(s) = a
ACTIONchange in state
SKILL POLICY
![Page 25: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/25.jpg)
What do I do in this state?
CONTINUOUS ACTION SPACES
∈
�(s) = a
QUERYrequest change in state
?
![Page 26: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/26.jpg)
HUMAN QUESTION ASKING
?
![Page 27: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/27.jpg)
HUMAN QUESTION ASKING
RESEARCH QUESTION
How do humans ask questions?
[Cakmak&Thomaz, HRI 2012]
![Page 28: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/28.jpg)
HUMAN QUESTION ASKING
[Cakmak&Thomaz, HRI 2012]
RESEARCH QUESTION
How do humans ask questions?
DOMAIN Four abstracted tasks
abstracted scoop&pour
![Page 29: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/29.jpg)
HUMAN QUESTION ASKING
RESEARCH QUESTION
How do humans ask questions?
DOMAIN Four abstracted tasks
DESIGN Observational study, task order counterbalanced
QUESTIONSDEMONSTRATIONS
x2
EXECUTION
x2
[Cakmak&Thomaz, HRI 2012]
![Page 30: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/30.jpg)
HUMAN QUESTION ASKING
DATA N=12, ~25 min, ~40 (SD=13) questions
VIDEO CODINGQuestion typesQuestion formsAccompanying actions
RESEARCH QUESTION
How do humans ask questions?
DOMAIN Four abstracted tasks
DESIGN Observational study, task order counterbalanced
[Cakmak&Thomaz, HRI 2012]
![Page 31: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/31.jpg)
Label queries
Demo queries
Feature queries 82%
4%
14%
QUESTION TYPES[Graesser, 1994]: Verifications, Example requests, Feature specifications
[Cakmak&Thomaz, HRI 2012]
![Page 32: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/32.jpg)
QUESTION TYPES
FEATURE RELEVANCE TEST
28%
FEATURE INVARIANCE TEST
35%
Sub-types of feature queries observed in humans
[Cakmak&Thomaz, HRI 2012]
![Page 33: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/33.jpg)
QUESTION TYPES
PARTIAL LABEL QUERY
60%
Sub-type of label queries observed in humans
[Cakmak&Thomaz, HRI 2012]
![Page 34: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/34.jpg)
QUESTION FORMS
VERBAL QUESTION FORMS [Kearsley, 1976]
DIRECT (98%)
INDIRECT (2%)I guess it doesn't matter
which hand I use.OPEN (13%)
CLOSED (85%)
SIMPLE (9%)How far do I
lower it?
EMBEDDED (3%)Is there an exact number
of small circles?
SPECIFIED-ALTERNATIVE (4%)Is color the only criterion or are
there other requirements?
YES/NO (81%)
TAG (2%)I turn it clockwise,
right?
SIMPLE (60%)Can I use other
colors?
INTONATED (19%)All shades of blue
are fine?
COMPLEX (1%)Does it matter what
I put into what?
[Cakmak&Thomaz, HRI 2012]
![Page 35: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/35.jpg)
USE OF EMBODIMENT
INSTANTIATIONS
26%
ICONIC GESTURES
8%
DEICTIC GESTURES
25%[Cakmak&Thomaz, HRI 2012]
![Page 36: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/36.jpg)
USE OF EMBODIMENT
Does orientation-around-x
matter?
NO EMBODIMENT WITH EMBODIMENT
“Does this orientation matter?”
[Cakmak&Thomaz, HRI 2012]
![Page 37: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/37.jpg)
EMBODIED QUERIES
Feature Query
Does this matter?
Label Query
Can I do this?
Demo Query
What do I do now?
![Page 38: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/38.jpg)
SKILL SEGMENTATION
humans segment skills into steps and ask questions about steps
![Page 39: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/39.jpg)
KEYFRAME-BASED SKILL LEARNING
Start recording
Stop recording
t=0
t=T
![Page 40: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/40.jpg)
Start like this
Then go here
Then go here
Finish like this
KEYFRAME-BASED SKILL LEARNING
k=0
k=1
k=2
k=3
![Page 41: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/41.jpg)
KEYFRAME-BASED SKILL LEARNING
![Page 42: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/42.jpg)
KEYFRAME-BASED SKILL LEARNING
![Page 43: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/43.jpg)
Obstacle
What is the purpose of queries?
KEYFRAME-BASED SKILL LEARNING
![Page 44: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/44.jpg)
KEYFRAME-BASED SKILL LEARNING
What is the purpose of queries?
Increase variance! ..with different query types.
![Page 45: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/45.jpg)
Feature Query
Does this matter?
Label Query
Can I do this?
Demo Query
What do I do now?
What is the purpose of queries?
Increase variance! ..with different query types.
KEYFRAME-BASED SKILL LEARNING
![Page 46: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/46.jpg)
LABEL QUERIES
Can I do this?
Which keyframes?How much variance? In which direction?
![Page 47: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/47.jpg)
?
LABEL QUERIES
query candidate
probability density of the point given data
increase in the varianceif the point is positive
Uk(s) = P(s|Dk)(var(Dk ∪ {s}) − var(Dk))
MAXIMIZE
α
![Page 48: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/48.jpg)
?
LABEL QUERIES
Uk(s) = P(s|Dk)(var(Dk ∪ {s}) − var(Dk))
MAXIMIZE
α
= 3 +1
nk(s − µDk
)TΣ−1Dk
(s − µDk) =
query candidate
Mahalanobis distance
![Page 49: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/49.jpg)
LABEL QUERIES
![Page 50: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/50.jpg)
PARTIAL-LABEL QUERIES
![Page 51: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/51.jpg)
Feature Query
Does this matter?
Label Query
Can I do this?
Demo Query
What do I do now?
EMBODIED QUERY TYPES
![Page 52: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/52.jpg)
FEATURE QUERIES
Does f1 matter at the
start?
high variance
f1
Do I have to have this value of
f1 at the end?
![Page 53: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/53.jpg)
FEATURE QUERIES
f1
Can f1 havethis value?
![Page 54: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/54.jpg)
FEATURE QUERIES
![Page 55: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/55.jpg)
EFFECTIVENESS OF QUERIES
RESEARCH QUESTION
How do different queries help learning?
![Page 56: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/56.jpg)
DESIGN Four conditions: unguided versus with queries
EFFECTIVENESS OF QUERIES
RESEARCH QUESTION
How do different queries help learning?
Label Feature Unguidednaive teacher
-VS-
Demo
![Page 57: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/57.jpg)
DOMAIN Bi-manual manipulation skills
DESIGN Four conditions: unguided versus with queries
EFFECTIVENESS OF QUERIES
RESEARCH QUESTION
How do different queries help learning?
![Page 58: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/58.jpg)
METRICS Applicability (in 50 tests) and Success (in 5 tests)
DOMAIN Bi-manual manipulation skills
DESIGN Four conditions: unguided versus with queries
EFFECTIVENESS OF QUERIES
RESEARCH QUESTION
How do different queries help learning?
![Page 59: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/59.jpg)
METRICS Applicability (in 50 tests) and Success (in 5 tests)
DOMAIN Bi-manual manipulation skills
DESIGN Four conditions: unguided versus with queries
EFFECTIVENESS OF QUERIES
RESEARCH QUESTION
How do different queries help learning?
DATA N=12, demonstrations for 5 mins
![Page 60: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/60.jpg)
All queries lead to more applicable skills
% o
f te
sts
96%100%88%
94%88%
94%
63%64%
Applicability of learned skills
No
quer
ies
Labe
l
Featur
e
Dem
o
No
quer
ies
Labe
l
Featur
e
Dem
o
POUR CLOSE-BOX
EFFECTIVENESS OF QUERIES
![Page 61: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/61.jpg)
All queries lead to more successful skills
% o
f te
sts
100%
80%
100%
80% 80%
60%53%
13%
Success of learned skills
No
quer
ies
Labe
l
Featur
e
Dem
o
No
quer
ies
Labe
l
Featur
e
Dem
o
POUR CLOSE-BOX
EFFECTIVENESS OF QUERIES
![Page 62: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/62.jpg)
INTUITIVENESS OF QUERIES
RESEARCH QUESTION
Can people easily answer different queries? Which do they prefer?
[Cakmak&Thomaz, HRI 2012]
![Page 63: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/63.jpg)
DOMAIN Goal-directed skills (pouring salt, cereal, coke)
DESIGN Within-subject study, 3 different query types
INTUITIVENESS OF QUERIES
RESEARCH QUESTION
Can people easily answer different queries? Which do they prefer?
Label Feature Demo
[Cakmak&Thomaz, HRI 2012]
![Page 64: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/64.jpg)
METRICSSubjective (perceived smartness, ease, informativeness)
and objective (time to answer)
DOMAIN Goal-directed skills (pouring salt, cereal, coke)
DESIGN Within-subject study, 3 different query types
INTUITIVENESS OF QUERIES
RESEARCH QUESTION
Can people easily answer different queries? Which do they prefer?
DATA N=18, 2 demonstrations, 2 queries in each condition
[Cakmak&Thomaz, HRI 2012]
![Page 65: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/65.jpg)
INTUITIVENESS OF QUERIES
72%
11%
17%
Feature queries
Smartest
28%
17%56%
Label queries
Easiest to answer
“Did not involve repeating the whole
process”
“Simon understood task constraints at a high level”
Subjective evaluation of different query types
[Cakmak&Thomaz, HRI 2012]
![Page 66: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/66.jpg)
INTUITIVENESS OF QUERIES
Objective evaluation of different query types
time (sec)
10 20 30 40
Question Answer
FEATURE
DEMO
LABEL
[Cakmak&Thomaz, HRI 2012]
![Page 67: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/67.jpg)
INTUITIVENESS OF QUERIES
Feature queries harder to interpret
[Cakmak&Thomaz, HRI 2012]
![Page 68: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/68.jpg)
SUMMARY
• Challenges with everyday users and the need for active learning
• Human question asking
• Human-like robot question asking
![Page 69: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/69.jpg)
Any queries?
[HRI 2012a] M. Cakmak and A.L. Thomaz. Designing Robot Learners that Ask Good Questions. International Conference on Human-Robot Interaction (HRI), 2012.
[HRI 2012b] B. Akgun, M. Cakmak, J.W. Yoo and A.L. Thomaz. Trajectories and Keyframes for Kinesthetic Teaching: A Human-Robot Interaction Perspective. HRI, 2012.
[TAMD 2010] M. Cakmak, C. Chao and A.L. Thomaz. Designing Interactions for Robot Active Learners. IEEE Transactions on Autonomous Mental Development, March, 2010.
Supported by NSF, ONR & Honda
![Page 70: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/70.jpg)
TOPIC 2
HUMAN-ROBOT HAND-OVERS
![Page 71: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/71.jpg)
HUMAN-HUMAN HAND-OVERS
WHY IS IT HARD?
NOT CONSCIOUS IN HUMANS HARD TO ARTICULATE “GOOD”
![Page 72: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/72.jpg)
HERB | INTEL OPEN-HOUSE | 2010
• Hand hanging in the air
• Multiple attempts
• Holding object together
• Robot waiting for a long time
• Need prompt and help from staff
• Pulling in different directions
• Need to change grasp
• Need to re-grasp after hand-off
COMMON PROBLEMS
![Page 73: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/73.jpg)
HERB 2.0 | RESEARCH@INTEL DAY | 2010
> Recorded hand-over attempts: 147 > Successful hand-overs to novices: 7
![Page 74: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/74.jpg)
PROBLEM 1 - COMMUNICATION OF INTENT
Visitor
HERB
![Page 75: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/75.jpg)
PROBLEM 1 - COMMUNICATION OF INTENT
![Page 76: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/76.jpg)
PROBLEM 2 - COMMUNICATION OF TIMING
Visitor
HERB
![Page 77: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/77.jpg)
NOT JUST HERB...
![Page 78: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/78.jpg)
PROBLEMS IN FLUENCY
ROBOT
HUMANIDEAL
Robot ready to hand
ROBOT
HUMANPROBLEM 2
ROBOT
HUMANPROBLEM 1
![Page 79: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/79.jpg)
COMMUNICATION OF INTENT
VERSUS
![Page 80: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/80.jpg)
COMMUNICATION OF TIMING
![Page 81: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/81.jpg)
PROPOSED SOLUTIONS
PROBLEM 1Spatial
Contrast
PROBLEM 2Temporal
Contrast
![Page 82: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/82.jpg)
HAND-OVER POSES FOR HERB
HANDING OVER?
![Page 83: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/83.jpg)
HAND-OVER POSES FOR HERB
Survey: What is the robot doing?
Holding the bottle
Looking at the bottle
Handing the bottle
Showing the bottle
Other
![Page 84: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/84.jpg)
HAND-OVER POSES FOR HERB
POSE PARAMETERS
Grasp locationObject tiltArm extension
![Page 85: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/85.jpg)
HAND-OVER POSES FOR HERB
Spatial contrast: Arm extended, object exposed
37% 40% 81% 92%
Poses that are picked as handing more (N=50)
![Page 86: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/86.jpg)
HAND-OVER *MOVEMENTS* FOR HERB
START END
Temporal contrast: Non-handing to handing
![Page 87: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/87.jpg)
EXPERIMENT
N=24 (9 female & 15 male, Ages: 20-45)
![Page 88: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/88.jpg)
INDEPENDENT VARIABLES
Sp
atial C
ontr
ast
Temporal Contrast
YES
NO
YES
NO
Within-groups, order counter-balanced
![Page 89: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/89.jpg)
EXTRANEOUS VARIABLE
BUSYAVAILABLE
Watching the robot Doing attention test
Between-groups
![Page 90: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/90.jpg)
FIXED VARIABLES
• Object transfer location
• Trajectory splining method
• Arm movement speed
![Page 91: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/91.jpg)
RESULTS
Temporal contrast improves fluency
Spatial contrast has no effect
![Page 92: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/92.jpg)
EFFECT OF TEMPORAL CONTRAST
ROBOT
HUMAN
Human wait time
Transfer happens
Human touches the object
![Page 93: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/93.jpg)
EFFECT OF TEMPORAL CONTRAST
ROBOT
HUMAN
Temporal
contrast
ROBOT
HUMAN
2.5 sec (SD=1.8)
1.4 sec (SD=0.9)
No
Temporal
Contrast
![Page 94: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/94.jpg)
EXAMPLES | NO TEMPORAL CONTRAST
2X
![Page 95: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/95.jpg)
EFFECT OF TEMPORAL CONTRAST
Early hand-over attempts
9 attempted
0 attemptedTemporal
Contrast
No
Temporal
Contrast
![Page 96: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/96.jpg)
EFFECT OF TEMPORAL CONTRAST
Missed responses in attention test
~3
~2Temporal
Contrast
No
Temporal
Contrast
![Page 97: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/97.jpg)
EXAMPLES | WITH TEMPORAL CONTRAST
2X
![Page 98: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/98.jpg)
EFFECTS OF SPATIAL CONTRAST
ROBOT
HUMAN
Robot wait time
Transfer happens
Robot ready to hand
![Page 99: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/99.jpg)
EFFECTS OF SPATIAL CONTRAST
NO EFFECT. WHY?
• Conveying intent was not an issue
• Intent was also conveyed by arm movement
VERSUS
![Page 100: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/100.jpg)
SUMMARY
Spatial contrast to communicate hand-over intent
Temporal contrast to communicate hand-over timing
![Page 101: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/101.jpg)
SUMMARY
HIDE & REVEAL
MOON ET AL. HRI 2014
![Page 102: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/102.jpg)
HUMAN-ROBOT HAND-OVERS
HOW TO PRESENT THE OBJECT?
to convey intent
![Page 103: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/103.jpg)
HUMAN-ROBOT HAND-OVERS
HOW TO PRESENT THE OBJECT?
to convey intent
to make it easy/intuitive to take for human
![Page 104: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/104.jpg)
HAND-OVER PARAMETERS
PLANNING/
OPTIMIZATION
LEARNING
FROM EXAMPLES
![Page 105: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/105.jpg)
LEARNING HAND-OVER CONFIGURATIONS
N=10 (8 male, 2 female)
3 good 3 bad examples
5 objects
1 2
3
4
5
1 2
3
4
5
![Page 106: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/106.jpg)
COLLECTED DATA
![Page 107: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/107.jpg)
EVALUATION
PLANNING
LEARNING
Which one did you prefer?
Which one looked more natural?
Which one was easier to take?
Which one was more appropriate?
![Page 108: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/108.jpg)
FINDINGS
PLANNING LEARNING
Preference 38% 62% χ2(1,N=50)=2.88, p=.09
Naturalness 36% 64% χ2(1,N=50)=3.92, p=.05
Practicality 46% 54% χ2(1,N=50)=0.32, p=.57
Appropriateness 38% 62% χ2(1,N=50)=2.88, p=.09
*
N=10 (6 male, 5 female), 5 objects
Subjective user evaluation
![Page 109: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/109.jpg)
FINDINGS
PLANNING LEARNING
Bottle 2 16
Mug 5 1
Notebook 2 7
Plate 6 3
Shaker 13 19
Number of events: bending, stepping forward, full arm extension
![Page 110: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/110.jpg)
TEXT
![Page 111: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/111.jpg)
FINDINGS
N=10 (8 male, 2 female)
61 pairwise comparisons
WHICH ONE IS BETTER?
![Page 112: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/112.jpg)
PREDICTIVE VARIABLES
% o
f co
mpari
sons
0
25
50
75
100
High
Low
Right
Left
Near
Far
Extended
Bent
![Page 113: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/113.jpg)
PREDICTIVE (LATENT) VARIABLES
% o
f co
mpari
sons
0
25
50
75
100
Default orientation Arm consistency Naturalness
![Page 114: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/114.jpg)
SUMMARY
PLANNING/
OPTIMIZATION
LEARNING
FROM EXAMPLES
Scalable/generalizable Preferred
HEURISTICS
![Page 115: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/115.jpg)
MORE PROBLEMS
Perception Object affordances Human-to-robot
WHAT? WHO? WHEN? WHERE? HOW?
![Page 116: Cakmak CSE571 March 2019 - University of Washington...HUMAN-ROBOT INTERACTION TAXONOMY H. Yanko and J. Drury "Classifying Human-Robot Interaction: An updated taxonomy" IEEE International](https://reader030.vdocuments.net/reader030/viewer/2022040821/5e6a148b36417d4d387455f8/html5/thumbnails/116.jpg)
THAT’S IT!
K. Strabala, M.K. Lee, A. Dragan, J. Forlizzi, S.S. Srinivasa, M. Cakmak and V. Micelli. Towards Seamless Human-Robot
Handovers. International Journal of Human-Robot Interaction. Vol. 1, No. 1, March, 2013.
S. Srinivasa, S. Berenson, M. Cakmak, A. Collet, M. Dogar, A. Dragan, R. Knepper, T. Niemueller, K. Strabala, M. Vande
Weghe, and J. Ziegler. HERB 2.0: Lessons Learned from Developing a Mobile Manipulator for the Home. Proceedings of
the IEEE, January, 2012.
M. Cakmak, S.S. Srinivasa, M.K. Lee, J. Forlizzi and S. Kiesler. Human Preferences for Robot-Human Hand-over
Configurations. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.
M. Cakmak, S.S. Srinivasa, M.K. Lee, S. Kiesler and J. Forlizzi. Using Spatial and Temporal Contrast for Fluent Robot-
Human Hand-overs. International Conference on Human-Robot Interaction (HRI), 2011.
M.K. Lee, J. Forlizzi, S. Kiesler, M. Cakmak, S.S. Srinivasa. Predictability or Adaptivity? Designing Robot Handoffs Modeled
from Trained Dogs and People. Late-breaking Report, International Conference on Human-Robot Interaction, 2011.