humanoides- osaka university

8/19/2019 Humanoides- Osaka University

1/46

CNR101x Week2 Handout

© 2016 Osaka University. All Rights Reserved.

1

Android Science

Distinguished Professor of Osaka University

Visiting Director & Fellow of ATR Hiroshi Ishiguro Laboratories

www.is.sys.es.osaka-u.ac.jp, www.geminoid.jp

1

Hiroshi

ISHIGURO

2

Robot Society


2/46



2

3

4

RobotSociety

Android

Android

Human


3/46



3

5

Why do we use thehumanoid?

Humans have brains that

recognize humans

The ideal interface for

humans are humans.

Studies on humans and

humanoids are studies on

principle of interface.

6


4/46



4

8

Near future society

supported by humanoids and android

Elderly care

Public places Learning

Department stores Stations

Humanoids that interact

with humans by using

multiple modalities


5/46



5

9

Research topics in the past and future

Neuro-

science

Robotics Pattern

Recognition

Information Society

1990 Cerebellum Mobile robot

Kansei

Model-based

Parametric

Web service

Sharing information

2000 Cerebrum Active visionBehaviorism

Learning

Learning XXX Social NetworkSociety on the net

2010 BMI Interaction

Development

Deep Learning Smart Phone

Real time small society2020 Frontal

cortex

Intention

Language

Intention

Consciousness

Society

Language

Non-parametric

Big data

Language

Personal Agent

Language processing

10

Android Science


6/46



6

Robotics for understanding humans

Appearance

Movement

Perception

Conversation

Learning and Development of the softwareBio-mimetic mechanisms

Human

11

12

Studies on humanlikeness


7/46



7

13

Development of interactive robots

Studies on Human-Robot Interaction300 action patterns and 700 transition

rules

14

Robots working in a shopping mallCombination of robots and sensor networks

Semi-autonomous robots


8/46



8

15

Development of child androids

Very humanlike appearanceBut not humanlike movements

内部メカニズム

女性アンドロイド16

Android: very humanlike humanoid robot


9/46



9

Subconscious and reactive movements

Subconscious movement Reactive movement• The subconscious movement gives human likeness to the android. However,

professionals find the difference.

• Reactive movements of a human is more complicated since it has mental states.

• Studies on cognitive and brain science improve the android.

17

Overcoming the uncanny valley

Neuroscience

study on

uncanny valley

The female android is not uncanny anymore.

Cognitive science and neuroscience with androids

Android Science: Integration of human studies and robotics

18

Similarity

F a m i l i a r i t y

Doll

Toy robot

Uncanny

valley

Human

Moving corpse100%

Child android

Adult

android


10/46



10

Science

Engineering PatternRecognition

MechanicalEngineering

Understanding onHuman behaviors

Hypothesis

and

verification

Android Science

Robotics and Cognitive Science

Scientific issue

Human likeness (appearance , movement, perception)

Engineering issue

Simple and interactive communication tasks

Systems

Engineering

Brain

science

Psychology

RoboticsDevelopment of mechanical humans

Cognitive

science

20

Practical application ofinteractive robots


11/46



11

21

Human archiving

Representation of human presence

Eternal life as a national treasure

22

Android vs. the person

Which presence is more strong?

What is the human presence?


12/46



12

23

Story-telling android

24

The original show window

Emotional states and 65 behaviors

Reactions to visitors

Practical role of the android


13/46



13

Sensor

25

Kinect

Camera

The control system

Sensor info

processionHow many people

How many peoplelook at F

Position of people

Gesture of People

Basic motionNeutral

(Waiting)

Looking around

Behavior model

Gazing someone

Looking around … Emotion

control O u t p u t

Emotion

-1HappyUnhappy

Arousal

1

1

PeaceDepression

Tension Excitation

Normal Happy Sad

Russel’s Model


14/46



14

27

Idol singer

Ideally beautiful singer

Many fans

28

Which is more beautiful, humans or androids?

What is the perfect idol singer?

What is beauty?


15/46



15

29

Sales clerk of the department storeDialogue system using tablet computers (without voice

recognition).

It handles more customers than human clerks.

30

Verbal conversationVoice recognition system and Cleverbot (Winner of

Loebner Prize in 2005 and 2006)

One question and one answer


16/46



16

31

Conversation among robots

They seem conversational.Feeling of intelligence from the robot conversation.

32

What is intelligence?

Is a human more intelligent than computers?

How can we define to think?

What is intelligence? What is thinking?


17/46



17

33

Selection of

related topics by

Google search

Understanding

concepts with

Wikipedia

Facial

expression

Gaze

control

Representation

of emotion

Emotion is the fastest way to connect among humansand robots.

Knowledge on the net for more intelligent conversation.

Intelligence in the cloud

Conversational pattern

34

Humanlike conversation

Practical teleoperation

and data collection


18/46



18

35

GeminoidTeleoperated android of an existing person

I n t e r n

e t

Operator

ジェミノイドを用いた会議36

Meeting with Geminoid


19/46



19

Comparison with video and speaker

Geminoid

Video

Speaker

The Geminoid has stronger presence than

others.

But it is more uncanny….

What is identity?38

Who has the identity? Can the human have identity?

Is there the peak for the

identity?

The identity is relative and it is

in the society.


20/46



20

39

Adaptation to

the different body

I n t e r n e t

40

Virtual tactile sensation

Operator


21/46



21

Body Ownership Model [Tsakiris M. et al 2005]

Visual

afference

Proprioceptiveafference

Motor

command

Motor

plan

Self-bodyRecognition

Arm

displacement

Arm

displacement

Comparator

Efference copy

Forward model

Body Ownership Model

during Geminoid Opereation

Visual

afference

Proprioceptive

afference

Motor

command

Motor

plan Arm

displacement

Arm

displacement

Comparator

Efference copy

Forward model

Geminoid

Body

Ownership

Illusion


22/46



22



Visual

afference


Motor

command

Motor

plan Arm

displacement

Arm

displacement

Comparator

Efference copy

Forward model

Geminoid

Body

Ownership

Illusion

Does the body ownership transferoccur in the absence of

proprioceptive feedback?

?

BCI Configuration

The subject imagines a left or right hand movement

Geminoid’s hands move according to the subject’s

brain activity

Amplifier

Signal

processing

Raw

data

gTEC BCI system

Motion command

Visual

Feedback

Geminoid system

classifierGeminoid

server


23/46



23

Experimental setup

The Procedure

Task Stimulus QuestionnaireTraining

12

Q: When the robot’s hand was injected, did you feel as if

your own hand was injected?

I didn’t feel at all 1 2 3 4 5 6 7 I felt very strongly


24/46



24

Evaluation method

SCR peak value after stimulus in each condition

were averaged and compared

Participants' score to the question in each condition were averaged and

compared

Questionnaire

Skin Conductance Response (SCR)

0

1

2

3

4

5

6

7

Still Match Raw

S c o r e

Results with 19 subjects

Significant difference in

questionnaire seen as

Match > {Still, Raw}

* ( p < 0.05)

Significant difference for SCR

seen as

Match > Still

* ( p < 0.05)

*

0

0.5

1

1.5

2

2.5

Still Match Raw

P

e

V

a

u

S

)

**

Questionnaire SCR


25/46



25



Visual

afference


Motor

command

Motor

plan Arm

displacement

Arm

displacement

Comparator

Efference copy

Forward model

Geminoid

Self-bodyRecognition

Body ownership

transfer

in the absence of

proprioception

50

Do we know ourselves?We do not know ourselves as much as we know others.


26/46



26

New media

that transfer our presence

to distant places

51

52

Conference with Geminoid


27/46



27

53

Experience is in our brain.

Experience is in others’ brain.

We, myself and the Geminoid can share the

experience.

What is experience?

54

Humanlike emotionand heart/mind


28/46



28

55

Emotional facial expression

Is it richer than human facial expression?

56

What is emotion?

Which is more emotional and more

sympathetic?

Can the android have human emotion?


29/46



29

57

Android theater

Moving and impressive theater withandroids.

It is world famouswith Hirata

with Hirata and Kuroki

58

Robot Theater 働く私Representation of humanlike heart


30/46



30

with Hirata and Kuroki

59

Robot Theater 森の奥

60

Can the android havehumanlike heart/mind?

Which is more strong?

What is heart/mind?


31/46



31

Development of

f ully autonomous robot

61

JST ERATO ISHIGURO

Symbiotic Human RobotInteraction

Project

2014-2020

62

I have developed the Geminoid originallyfor gathering data to make it autonomous.

Development of autonomous robots

I n t e r n e t

OperatorInterlocutor

Teleoperated

Android(Geminoid)

The teleoperated androids and robots

AutonomousAndroid

Autonomous androids and robots


32/46



32

63

Big data approach for developing autonomous robots

I n t e r n e t

OperatorInterlocutorTeleoperated

Android

(Geminoid)

Speech and gesture

recognition

Utterance andbehavior generation

Hierarchical

models ofutterance andbehavior

generation

Behavior

Intention

Desire

Robot

Enhancement of the internal humanlikeness

Role assignment

in particular situations

Development of an acoustic model for

effective speech recognition by

Deep LearningCollect a great deal

of conversational

patternsincluding movements and

emotional expressions

Classification of

the patterns

64

Hierarchical models of

utterance and behavior generation

Behavior

Intention

Desire

Real

worldInternal

representation

Sensing

Action

Sensing(Prediction)

Action

(Revision)

Robot

Hierarchical models studied for decision-making in mobile

robotics.

The models are based on the idea accepted in cognitive science

and artificial intelligence.

The desires generate intentions, and intention generate

behaviors and utterances.


33/46



33

65

Acquisition of the Hierarchical Models

Behavior

Intention

Desire

Behavior

Intention controls activation or selection of behavior.

Desire controls activation or selection of intention.

Machine learning techniques classify collected data with

intention flags given by the operator and find paths for satisfying

intentions.

BehaviorBehavior

Behavior

BehaviorBehavior

Behavior

Behavior

Behavior

BehaviorBehavior

BehaviorBehavior

Behavior

Behavior

BehaviorBehavior

Intention Intention

Desire Desire

IntentionIntention

Intention

66

Inference of people’s desires and

intentions

based on the hierarchical models Robots need to infer other’s intention and desire in social situations for having

proper conversations.

The robots assume the hierarchical models not only for other robots but alsofor humans, and infer humans’ intentions and desires.

What is mind? What is consciousness?

Geminoid

Geminoid

Human

Speech and gesture

recognition

Utterance and

behavior generation

Hierarchical

models ofutterance and

behaviorgeneration

Behavior

Intention

Desire

Robot

Speech and gesture

recognition

Utterance and

behavior generation

Hierarchical

models ofutterance andbehaviorgeneration

Behavior

Intention

Desire

Human


34/46



34

67

The minimal human

68

How do we recognize

humans and geminoids？


35/46



35

69

How do we recognize

the human presence with Telenoid?

70

Human has imagination

Humans interact withothers by using the

imagination.

The minimum design

maximizes the imagination.


36/46



36

71

Telenoid

I n t e r n e t

The imagined face is mentally projected onto the neutral

appearance.

72

Communication service for elderlies


37/46



37

73

Field tests with Telenoids

Practical service for elderly care.In Denmark and Japan.


38/46



38

75

Portable media

for transmitting human presenceNew information media after smartphone

From voice and visual to human presence

76

The 3G prototype


39/46



39

77

78

The minimum condition

to feel human presenceIt requires two modalities.

Hugvie: the minimum media.

Smartphone holder

Talk by phones


40/46



40

79

Relax effect of Hugvie

Cortisol decrease by using Hugvie

p


41/46



41

81

With hugvies

82

More than two modalities

Voice + Tactile sensation

Appearance + Tactile sen.

Smell + Tactile sensation

Voice + Smell

What do we need for feeling human

presence?

0 1 2 3 4 5

Number of modalities

Strength

of the feeling


42/46



42

83

Humanlike Mechanism

Engineering explanation

of Yuragi formula

activity x f xdt

)()(x f x dt

d

YuragiOptimization measureControl model with attractors

Yuragi

formula

Currentcontrol

Control model


43/46



43

Engineering explanation

of Yuragi formula activity x f x

dt )()(x f x

dt

d

u( x ) does not have to be precise or

accurate

activity Attractor

State valuable x

activit x u

)(

u( x )・activity is minimized by theyuragi formula

When activity is small, Yuragi η becomes relatively large, and it searches semi-

optimal solution based on Yuragi. Then, activity becomes large and it stays there.

It can be applied for complicated cases where proper models do not exist.

It does not compute all combinations, but it rather searches by Yuragi anddrastically reduce the computational cost/energy.

dx

x du x f

)()(

Potential

u( x )

YuragiOptimization measureControl model with attractors

Yuragi

formula

Current

control

Control model

85

86

Control of complicated robots based

on the biological principle

Bacteria robot with one motor

and one sensorRobotic arm that has humanlike bone structure

and muscle arrangement


44/46



44

87

Simulation ofhumanlike development

with Asada (Osaka Univ.)

Humanlike developmental mechanism

• Yuragi simulates the infant

developmental process.

with Asada (Osaka Univ.)

88

U-shape development of infant

Variations

of themovements

Age

General

movements

Identification of the

elemental behaviors

Combinations of theelemental behavior


45/46



45

89

Difference between humans and robot?

Can we distinguish them with

the fresh body?

Why do humans want to be

humans?

Someday, the boundary maydisappear. Is the boundary

necessary?

90

What is the definition of Human?

It is changing.

Are humans always more valuable than robots?

Do humans have the unlimited value to survive?


46/46


91

Android

Android

Human

Robot society

Ubiquitous

systems

Cognitive science

Standardization

Economy

Marketing

Robotics

Sociology

Field test

Complex systemsChaotic systems

Biology

Material

Engineering Sensor

Actuator

Artificial Life

Pattern

Recognition

Systems

Engineering

Neuroscience

User

Interface

Media & Art

Philosophy

Psychology

humanoides- osaka university

Documents