Toward a Science of Robotics:Goals and Standards forExperimental Research

Leila TakayamaHuman-Robot InteractionResearch Scientist

Scientific Principles

•Hypothesis testing•Observable, empirical, and

measurable evidence•Reliable•Reproducible and falsifiable

Science & Technology

Variable-based Research

X → Y

“analog, keyboard and receiver input, high fidelity,two-way processing technologies”

Examples of Variables

•System complexity•Interactivity•Similarity to humans•Task types: collaborative, competitive•Synchronicity of interaction•Collocation

Study 1 Self Extension into Robots

Research Questions– What aspects of human-robot interface

design affect feelings of attachment, trust,control, responsibility, and agency inhuman-robot interaction?

– Does one’s sense of self-extension increasewhen a robot is built by the user?

– Does one’s sense of self-extension differbetween more or less anthropomorphicrobots?

Manipulations

Factor 1 (between): car vs. robot

Factor 2 (between): Use of their assembledrobot/car or a preassembled robot/car

Factor 3 (between): assembly vs. noassembly

HypothesesRobot form

Humanoid form is a strong cue ofidentity– H1. People will self extend more into the

car robot than the humanoid.

People extend their positive selfconcept into self-extended objects

– H2. People will prefer the personality ofthe car robot over the humanoid.

9

HypothesesRobot assembler

Building an object promotes self extension– H3. People will self extend more into a

robot they assemble than a robotassembled by another.

People extend their positive self concept intoself-extended objects

– H4. People will prefer the personality of arobot they assemble over a robotassembled by another.

10

Procedure• Participants fill in pre-questionnaire

• Participants given instructions anddiagrams on how to assemble robot

• Participants assemble robot– (M=9 min 49 sec, SD=3 min, 34 sec)

• Participants turn on robot and test it– Tethered control with on/off button

11

Procedure

12

Task game

• Goal is to collect most points in 10 minutes• Bombs sometimes explode when touched• Bomb detonations deduct 30 seconds• Bomb number and time controlled• Questionnaire

Assembler Manipulation

•Manipulating assembler– Self: Built a robot, operated same robot– Other: Built a robot, participants told they

needed to operate a different, identicalrobot

•In fact, all participants operated therobot they built

13

Robot Form Manipulation

Measures:Self extension

Trait overlap– Personality similarity of self and other

•Galinsky and Moskowitz– Overlap in concepts of self and human other

•Kiesler and Kiesler– Self extension into objects

15

Self Other

Measures:Self extension

Determining trait overlap– Thirty item modified Wiggin’s personality test

•Completed by participants about themselves before task•Completed by participants about robot after task

– Delta of items calculated, summed to index•(Cronbach’s α=.86)

– Smaller scores indicate greater overlap ofconcepts of self and robot

16

Measures:Self extension

Self reports• 10 point scales asking about “the device you

guided through the minefield”• Robot control (α=.83)

– Who was more responsible for your generalperformance on this task?

– Who had more control over your generalperformance on this task?

• Sense of team– “I felt that the robot and I were a team.”

17

Measures:Robot personality

•Robot friendliness– Nine item index (α=.90)– cheerful, enthusiastic, extroverted

•Robot integrity– Five item index (α=.73)– Honest, reliable, trustworthy

•Robot malice– Five item index (α=.74)– Dishonest, unkind, harsh

18

Results:Self extension

Trait overlap

19

F(1, 52)=4.04, p<.05, partial η²=.13

Greater trait overlap with car robots than humanoids

Results:Self extension

Robot control

20

Greater relative control attributed to humanoids thanto cars

F(1, 52)=5.47, p<.05, partial η²=.10

Results:Self extension

Sense of team

21

F(1, 52)=8.34, p<.01, partial η²=.14

Self-assembly participants felt more like a team with the robotthan did other-assembly participants

Measures:Robot Personality

Robot friendliness

22

F(1, 52)=4.25, p<.05, partial η²=.08 F(1, 52)=4.23, p<.05, partial η²=.08F(1, 52)=4.23, p<.05, partial η²=.08η²=.08η²

Car robots were friendlier thanhumanoids

Self-assembled robots were friendlierthan robots assembled by others

F(1, 52)=4.25, p<.05, partial η²=.08η²=.08η²

Measures:Robot Personality

Robot integrity

23

Car robots were rated as having more integrity than humanoids

F(1, 52)=4.20, p<.05, partial η²=.08

Measures:Robot Personality

Robot malice

24

F(1, 52)=8.94, p<.01, partial η²=.15 F(1, 52)=4.78, p<.05, partial η²=.08

Humanoid robots were moremalicious than cars

Robots assembled by otherswere more malicious than

self-assembled robots

F(1, 52)=8.94, <.01, partial η²=.15η²=.15η² F(1, 52)=4.78, p<.05, partial η²=.08η²=.08η²

Summary of Results

H1. People will self extend more into the carrobot than the humanoid.

H2. People will prefer the personality of thecar robot over the humanoid.

25

Summary of Results

H3. People will self extend more into a robotthey assemble than a robot assembled byanother.

H4. People will prefer the personality of arobot they assemble over a robot assembledby another.

26

Design Implications

Goal-specific guidelines

– No form, assembly experience is uniquelyoptimal

– Desirability of self extension informs design

27

Design Implications

•When self extension is desired– Tele-operated robots as media, human

representations•Medical care, remote therapy

– Non-humanoid form– Promote pre-mission interaction

•Assembly, customization

28

Design Implications

•When self extension is undesirable– Robots in hostile environments, likely

failures•Search and rescue

– Humanoid form– Minimize pre-mission interaction

•Identical but different robots– Change robot’s name

•Altered robots– Change voice, appearance

29

Limitations and Next Steps

• Broader population• Outside the lab• Using other robots

• Long-term interactions• Long-term effects• Balancing needs of people operating and

encountering robot

30

Study 2Disagreeing Robots

Why would a robot everdisagree with a person?

Research Questions

•What influences an interface’s point ofinteraction? Body location? Voicelocation?

•(How) do politeness strategies fromhuman-human interaction informhuman-computer interaction?

Design Questions

•What influences a robot’s point ofinteraction?

•Where should speakers be placed?•(How) can computer agents influence

human decisions, using effectivepoliteness strategies?

HypothesesH1. People will change their decisions more often

when the robot disagrees with them than when italways agrees with them, even with identicalsubstantive content.

H2. People will feel more similar to (H2a) and morepositively toward (H2b) the agreeing robot than thedisagreeing one.

H3. A disagreeing voice coming from a separatecontrol box will be more acceptable than adisagreeing voice that came from the robotic body.

Study Design (N=40)

20 men and 20 women, balanced across conditions

Disagree 60%

Disagree 0%

Voice location:in box

Voice location:on robot

Between-participants

Procedure

• Write down decisions about desert survival items• For each item

– Tell robot which item to retrieve– Robot responds with survival item information

and judgment about decision– Tell robot which item to retrieve

• Write down final decisions about survival items– These final ratings will be “evaluated”

• Fill out paper questionnaire

Desert survival task

You are one of the members of a geology clubthat is on a field trip to study unusual formationsin the New Mexico desert. It is the last week inJuly. You have been driving over old trails, farfrom any road, in order to see out-of-the-wayformations. At about 10:30 A.M. the speciallyequipped minibus in which your club is ridingoverturns, rolls into a 20-foot ravine, and burns.The driver and professional advisor to the clubare killed. Both of you are relatively uninjured…

Desert survival task

Rank the following items according to their importance to your survival, startingwith 1 for the most important one and proceeding to 12 for the least important one.

______ magnetic compass______ 20-ft by 20-ft piece of heavy-duty, light-blue canvas______ book, Plants in the Desert______ rearview mirror______ large knife______ flashlight (four-battery size)______ one jacket per person______ one transparent, plastic ground cloth (6-ft by 4-ft) per person______ .38-caliber loaded pistol______ one 2-quart plastic canteen per person, full of water______ accurate map of the area______ large box of kitchen matches

Experiment Set-up

Manipulation: Robotdisagreement

Statement Examples

1.

Description

of selected

item

The knife could be helpful in

cutting down stakes to build a

solar still or to build shelter. It

could also assist in cutting down

firewood for a fire.

2.

Judgment:

disagreeing

or agreeing

That is not as

good as…

That is a better

choice than…

3.

Description

of

alternative

item

The pistol, which could be good

for signaling for help. It could

provide an alternative noise

source if your voice is weak due

to dehydration.

4. Request

for final

selection

Which do you choose?

or

MeasuresBehavior• Number of decisions changed

Attitudes• Perceived agreeableness of robot

(2 items, Cronbach’s α=.69)

• Perceived similarity of robot to self(4 items, Cronbach’s α=.94)

• Liking of the robot(8 items, Cronbach’s α=.75)

Perceived robotagreeablenessagreeableness

Perceived similarity to robot

People changed their minds

People like disagreement tocome from elsewhere

Checking against hypothesesH1. People will change their decisions more often

when the robot disagrees with them than when italways agrees with them, even with identicalsubstantive content.

H2. People will feel more similar to (H2a) and morepositively toward (H2b) the agreeing robot than thedisagreeing one.

H3. A disagreeing voice coming from a separatecontrol box will be more acceptable than adisagreeing voice that came from the robotic body.

Theory-orientedInterpretations

•Politeness: distancing•Disembodiment•Perceived source

– Two separate agents: Thinker and doer– Single distributed agent

Design-oriented Implications

• Voices can be more evocative than robot bodies• Agents can be sources of judgment and opinions• People are sensitive to disagreements• Disagreement undermines feelings of similarity• When agreeing 100% of the time, put the voice on the

robot body• When disagreeing (sometimes), put the robot voice

elsewhere

Validity

• face validity: how reasonable a measureseems to be for its concept

• content validity: how thoroughly a measureaddresses the breadth of a concept

• construct validity: how much a measurecausally relates to other variables withinone’s theory

• external validity: how generalizable theresults will be to other systems and contexts

Experiment Designs

•Keep it simple•Ceteris paribus•Random assignment to conditions•Balancing•Standardized tasks and measures•Behavioral and attitudinal measures•Sample representativeness

Stats

•Use with caution!•Especially with

statistical modeling

Reporting Studies

• Research questions andhypotheses

• Statistical significance• Reproducible methods• Discuss limitations• Thoroughly review

related work• Reduce bias in language• Clear labeling• Define terms

Experiment Work Practices

•Pilot stimuli, measures, procedures withmultiple types of pilot participants

•Identifying the important variables andtheir relationships (grounded theory)

Sharing artifacts and code

www.willowgarage.com

Thanks!

Victoria Groom, Clifford Nass

Claudia Jimenez, Alison King, Morgan Ames, CourtneySchultz, Paloma Ochi, Jessica Yuan

Contact: Leila Takayamatakayama@willowgarage.com

Data Frame Model