Atsuo TakanishiDept. of Modern Mech. Eng./Humanoid Robotics Institute, Waseda University

Humanoid RoboticsHumanoid Roboticsat Waseda Universityat Waseda University

Tutorial on Humanoid TechnologiesTutorial on Humanoid Technologies

The 2The 2ndnd EC FP7 RoboSoM Project MeetingEC FP7 RoboSoM Project MeetingFebruary 18, 2010February 18, 2010

Prof. Ichiro Kato Prof. Ichiro Kato (1925(1925--1994)1994)

WasedaWaseda’’s Achievementss Achievementsin Humanoid Roboticsin Humanoid Robotics

led by the late Prof. Ichiro Katoled by the late Prof. Ichiro Kato

WABOT-1 (1973) WABOT-2 (1984)

Why Humanoid Robotics?Why Humanoid Robotics?To Build Human Model to Understand Human from Robotics View Point: “Robotic Human Science”To Make Robotics as One of the Social Technologies based on the Human Model to Support Elderly Dominated Society and to Support Education, Etc.Grand Challenge for Roboticists: A human is considered to be an extreme robot both in motional functionality and in intelligential functionality.

Humanoid FamilyHumanoid Familyin WABOTin WABOT--HOUSE ProjectHOUSE Project

WABIANWABIAN--2 as a Robotic Human Simulator and2 as a Robotic Human Simulator anda Measuring Device for Quantitative Evaluationa Measuring Device for Quantitative Evaluation

of Handicapped/Elderly Supporting Machines/Robotsof Handicapped/Elderly Supporting Machines/Robots

height : 1.5 [m]weight : 55 [kg]

Total DOF : 41leg : 6×2waist : 2trunk : 2arm : 7×2neck : 3

Yu Ogura TeamYu Ogura Team

2 DOF

3 DOF

1 DOF

3 DOF

22--DOF PelvisDOF Pelvis

New LegNew Leg--Body Mechanism Body Mechanism Inspired by Human WalkingInspired by Human Walking

Modeling of Pelvis Motion

2 DOF hips

•Stretch walking •Obstacle avoidance

2DOF

1DOF

3DOF

3DOF

Pelvis-Waist Mechanism:2DOF

Most Tough Parts in Design:Most Tough Parts in Design:WaistWaist--PelvisPelvis--HipHip--KneeKnee

WABIANWABIAN--2 Designed Based2 Designed Basedon the Human Dimensionson the Human Dimensions

More HumanMore Human--like Walking by like Walking by WABIANWABIAN--2R with a 50cm2R with a 50cm--StepStep

0.50[m/step], 0.96[s/step]

0.40[m/step], 0.96[s/step]

HumanHuman--Like Outdoor Walking Like Outdoor Walking

Two Peaks in Vertical Floor Two Peaks in Vertical Floor Reaction ForceReaction Force

Human WalkingHuman Walking

Knee Bended Knee Bended WalkingWalking

Knee Extension WalkingKnee Extension Walking

WABIANWABIAN--22’’s Walking Assisted by s Walking Assisted by Hitachi Walker for Elderly/HandicappedHitachi Walker for Elderly/Handicapped

Bending 0.2[m] Knee stretched 0.2[m]

Real World Simulations of a human assisted by the Walker

9th,FEB,2007 ２００６年度 修論発表

■ Optimal Pattern

Subject

Applications of WasedaApplications of Waseda’’s Bipedals BipedalWalking Robot TechnologyWalking Robot Technology

METIMETI HRPHRP--11ZMP PINO2ZMP PINO2 TMSUKTMSUK／／Waseda WLWaseda WL--1616

SONY QRIOSONY QRIO

TMSUKTMSUK／／SANYO BanryuSANYO Banryu

SHINPOSHINPO「「新歩新歩」」: A Brother of WABIAN: A Brother of WABIAN--2 2 Walking in Niigata Natural Science Walking in Niigata Natural Science Museum Resulting of Collaboration with Museum Resulting of Collaboration with TMSUKTMSUK

Samurai Humanoid Samurai Humanoid KIYOMORIKIYOMORI::22ndnd Brother of WABAINBrother of WABAIN--2 Visited Munakata Shrine 2 Visited Munakata Shrine

Prayed for the Safety of All the Robots in the WorldPrayed for the Safety of All the Robots in the World

33--Moment Compensatory Bipedal Moment Compensatory Bipedal Walking Control Extending the ZMP Walking Control Extending the ZMP CriteriaCriteria

Y

PP

RR

YY

PP

RR

m ( ) ( )

{( ) }

ii

All Particles

i ZMP i

Fk ZMPk

All Points

k k

∑

∑

− × −

− − × + =

r r r G

r r F M

&&

0

GravityGravity

FrictionFriction

March 18, 2005March 18, 2005

How to walk with balance

ZMP: a point where the total forces and moments acting on the robot are zero

ZMP trajectory: set arbitrarily within the support polygon

Stable region

Compensatory motion to keep the balance, computed using ZMP equation

Foot trajectory ZMP trajectory

Stability criterion for biped robot based on ZMP (Zero Moment Point)

March 18, 2005March 18, 2005

ZMP Equation and the Coordinate Systems

XYZO −

XYZO −

m ( ) ( )

{( ) }

ii

All Particles

i P i

Fk Pk

All Points

k k

∑

∑

− × −

− − × =

r r r G

r r F M

&&

+ + T 0

m ( ) ( )

{( ) }

ii

All Particles

i ZMP i

Fk ZMPk

All Points

k k

∑

∑

− × −

− − × + =

r r r G

r r F M

&&

0

ZMP Equation

m ( ) {

2 ( )}

{( ) }

ii

All Particles

i ZMP i

i i i

Fk ZMPk

All Points

k k

∑

∑

− × + −

+ × + × + × ×

− − × + =

r r r Q G

r r r

r r F M 0

&& &&

& &ω ω ω ω

March 18, 2005March 18, 2005

Iterative Computation Algorithmto Converge ZMP/Moment Error

Compute walking pattern using compensatory waist motion

Compute moments Mgenerated by lower-limbs

Compute compensatorytrunk and waist motion

using linear model (FFT&IFFT)

Compute moment errors (eM) between calculated ZMP and planned ZMP

eM < εM ?Yes

No

Compute walking pattern using initial waist motion

Complete walking pattern

Initial parameters;Foot trajectory, Initial trunk and waist trajectories,ZMP trajectory, and εM (a tolerance level moment error)

nLL EMM +=

moment errors (eM)

)1(1 −− ⋅+= nnn Gain eMEE

Iterative computationto derive motion pattern

WWTT yxyx ,,,

WABIANWABIAN--2R Goes to Streets, 2R Goes to Streets, but Many Problems!but Many Problems!

Special Economical Zone for Robot Test and Experiment in FukuokaSpecial Economical Zone for Robot Test and Experiment in Fukuoka

We need more efforts for higher usability in the real worldWe need more efforts for higher usability in the real world

Open Experiment at Okubo Campus of Waseda in 2006

Biped Robot That CanBiped Robot That CanCarry a Human: WLCarry a Human: WL--1616

WLWL--16 Goes to the US16 Goes to the USCarrying 170 People in NextFest 2007Carrying 170 People in NextFest 2007

Towards Vocal Humanoid Towards Vocal Humanoid Robot: WT Series (2000Robot: WT Series (2000-- ))

Human Vocal OrgansHuman Vocal Organs

Robot Vocal OrgansRobot Vocal Organs

Jump!Jump!

Kotaro Fukui TeamKotaro Fukui Team

Sound SourceSound Source

Vocal CordsVocal Cords

Out PutOut Put

LipsLips

Soft PalateSoft Palate

TongueTongue

Airflow from LungsAirflow from Lungs

Palate Palate

Nasal Cavity Nasal Cavity

WT-5’s Mechanism

xx

zz

yy

Human Vocal Cords

戸幾一郎：発声機構の面よりみた喉頭の病態生理、耳鼻臨床（戸幾一郎：発声機構の面よりみた喉頭の病態生理、耳鼻臨床（19661966））

※※

Structure of Vocal CordsStructure of Vocal Cords

Vocal Fold VibrationVocal Fold Vibration

Lower PartLower Part

Upper PartUpper Part

Cover Cover (Nonmuscle Tissue )(Nonmuscle Tissue )

BodyBody(Muscle Tissue )(Muscle Tissue )

Phase Difference in Glottal Closure Phase Difference in Glottal Closure Between Upper Part and Lower PartBetween Upper Part and Lower Part

WT-5’s Vocal Cords

35353030

5050

xxyyｚｚ

1515 2424

ｚｚ

frontfront sideside backbacktoptop

Developed New Vocal Cords Mimicking Developed New Vocal Cords Mimicking Human Biomechanical StructureHuman Biomechanical Structure

2222

1616

yyzz

xx

Vocal Fold Vocal Fold VibrationVibration

** Thermoplastic Rubber by Kuraray CoThermoplastic Rubber by Kuraray Co

Using the Thermoplastic Rubber Using the Thermoplastic Rubber ““**SeptonSepton””Same Size as Human (Adult Male)Same Size as Human (Adult Male)

Voiced/Unvoiced Sounds Switching Mechanism

CloseClose OpenOpen

OpenOpenCloseClose

Glottal Opening/Closing Control Mechanism by Glottal Opening/Closing Control Mechanism by using Arms Mimicking Human Arytenoid Cartilagesusing Arms Mimicking Human Arytenoid Cartilages

xx

yyzz

Modeling of Elastic Tongue

Elastic Tongue ShapeElastic Tongue Shape

Rubber( EPDM )Rubber( EPDM )

Applicable to All Elastic BodyApplicable to All Elastic BodyApplicable to All Elastic Body

ControlControlPointPoint

Modeling of Tongue asModeling of Tongue asConnection ofConnection of Rigid LinksRigid Links

RigidRigidLinksLinks}}

ControlControlPointPoint

SimulateSimulate

Modeling of Elastic Tongue

( )11, yxx

1l

nq

y

Free End

α

1q

nl( )nn yx ,

Virtual Link

1k

1+nk

Virtual Link

nk

2q2q2q2k

yx FFF ,

⎩⎨⎧

−=−=

yny

xnx

PyPx

δδ

Virtual DisplacementVirtual Displacement by Force F,by Force F,( )yx δδ ,

Potential Energy (by Potential Energy (by Virtual Work PrincipleVirtual Work Principle))

yyxx

n

iii FFqkqkU δδπ

+++⎟⎠⎞

⎜⎝⎛ −= ∑

+

=

1

2

22

11 21

221

Referred by HookeReferred by Hooke’’s Laws Law

(3)(3)

(4)(4)

Kinetic Energy (Stationary)Kinetic Energy (Stationary)LagrangeLagrange‘‘s Equations of Motion ( )s Equations of Motion ( )UTL −=

0=T

(5)(5)

( )nqqq ,,, 21 …Link AnglesLink Angles

NewtonNewton--Raphson MethodRaphson Method

Towards Vocal Humanoid Towards Vocal Humanoid Robot: WT SeriesRobot: WT Series

Human Vocal OrgansHuman Vocal Organs

Robot Vocal OrgansRobot Vocal Organs

Jump!Jump!

Kotaro Fukui TeamKotaro Fukui Team

Flutist Robot for SimulatingFlutist Robot for SimulatingHuman Flute Playing:WFHuman Flute Playing:WF--44

(1990(1990-- ))

GifuGifu--Waseda WABOTWaseda WABOT--HOUSE ProjectHOUSE Project

Mr. Wakamatsu:Mr. Wakamatsu:Technical AdvisorTechnical Advisor

W a s e d a U n i v e r s i t y第25回日本ロボット学会学術講演会 2007年9月14日

ソリス ホルへ

Lips：3-DOFsy

z

x

Waseda Flutist No.4 Refined IVLips Mechanism Design

z

W a s e d a U n i v e r s i t y第25回日本ロボット学会学術講演会 2007年9月14日

ソリス ホルへ

Waseda Flutist No.4 Refined IVLips Mechanism Motion

W a s e d a U n i v e r s i t y第25回日本ロボット学会学術講演会 2007年9月14日

ソリス ホルへ

Tongue

Oral cavity

yx

x

z

Film of SEPTON

z

Waseda Flutist No.4 Refined IVTonguing Mechanism Design

Tongue：1-DOF

W a s e d a U n i v e r s i t y第25回日本ロボット学会学術講演会 2007年9月14日

ソリス ホルへ

Waseda Flutist No.4 Refined IVTong Mechanism Motion (Tonging)

Waseda University

83 [m

m]

45 [mm]

x y

z

Vibrato Mechanism

Human-like Vibrato Mechanism

Human Vocal Cords

ProfessionalIntermediate

Vibrato Mechanism

Low AmplitudeHigh Amplitude

4 [Hz], A4x

y

Mukai(1987)

W a s e d a U n i v e r s i t y第25回日本ロボット学会学術講演会 2007年9月14日

ソリス ホルへ

Lung Mechanism

ShaftShaft

AC ServomotorAC Servomotor

LinkLink

yy

zz

413 [mm]413 [mm]

398

[mm

]39

8 [m

m]

yy

zz

Takanishi Lab. 14th Oct. 2002The Robotics Society of Japan

Finger Mechanisms

DC Motor

pulleyOuter TubeTeflon-lined Wire (Chukoh Chemical Industries LTD)

Link mechanism

Supported by SokidWorks K.K

VV : Total Sound Volume: Total Sound Volume

HH : Average of Harmonics: Average of Harmonics

HsHs : Average of Semi: Average of Semi--HarmonicsHarmonics

HdHd : Even Har. : Even Har. –– Odd Har.Odd Har.

ItemsItems

W1, W2 : WeightinsW1, W2 : Weightins

FF(Note Pitch)(Note Pitch) = = VV + w+ w1 1 ×× ((HH--HsHs) + w) + w2 2 ×× HdHdRR RRR [Pitch Range] = {Low, Middle, High}R [Pitch Range] = {Low, Middle, High}

What is good flute sound?:What is good flute sound?:Evaluation IndexEvaluation Index

WFWF--44’’s Flute Play:s Flute Play:The Flight of the BumbleThe Flight of the Bumble--BBeeee

(Rimsky(Rimsky--Korsakov)Korsakov)

Duet between Human and WFDuet between Human and WF--44

Saxophonist Robot: WASSaxophonist Robot: WAS--22

Part Degrees of Freedom

Pump 1 DOFValve 1 DOFLip 3-DOFTongue 1-DOFFingers 16-DOFs

Total : 22 DOFsTotal : 22 DOFs14

0014

00

WASWAS--22

Lip Lip & Tongue& Tongue

FingersFingers

PumpPump

ValveValve

New Mouth & Hand MechanismNew Mouth & Hand Mechanism

TEAM FLUTE

Waseda universityRSJ2009New Mouth Mechanism

Overview

Part No. of Degree

Upper lip 1

Sideway (1)

Lower lip 1

xyz

WAS-2

TEAM FLUTE

Waseda universityRSJ2009Development of a Human-Like Hand

Video

Demo Play of WAS-2：Sing Sing Sing

Emotion Expression Humanoid Emotion Expression Humanoid EYEEYE--ChanChan::WEWE--4RII for Modeling Human Mind4RII for Modeling Human Mind

PartNeckEyes

EyelidsEyebrows

LipsJaw

Arms

Lung

68

34

411

18

DOF

Waist 2

Total 59Hand 12

WE-4 (Waseda Eye No.4)Expressions

Eyebrows

Eyelids

Facial Color

Lips

Voice

Neck

Waist

Lung(Breath)

Sensors

Visual(CCD Camera)

Auditory(Microphone)

Olfactory(Gas Sensor)

Tactile(FSR)

Temperature(Thermistor)

Control System and PersonalityControl System and Personality

Total Personality

Intelligence

Environment

RecognitionBehavior

EmotionSensing

PersonalityExpression Personality

SensingMotion Reflex

Equations of EmotionMt+Δt = Mt + ΔM

Mt = (at, pt, ct)ΔM = (Δa, Δp, Δc)

at : Activationpt : Pleasantnessct : Certainty

arousal

asleep

pleasant unpleasant

certain

uncertain

Mt

Mt+ΔtΔM

Equations of EmotionEquations of Emotion

Pleasantness

Activ

ation

Cer

tain

ty

Happiness

Surprise

Happiness

Disgust

Sadness

Fear

Sleep

Sleep

Neutral

Neutral Anger

*Paul Ekman, Wallace V. Friesen: Facial Action Coding System (1978 )

Emotion MappingEmotion Mappingin Mental Spacein Mental Space

E = {Happiness,Anger,Surprise, Sadness,Fear,Disgust,Neutral}

emotion ∈ E

From Eq. of MotionFrom Eq. of Motionto Eq. Emotionto Eq. Emotion

M ： Emotional Inertia Matrix（情動慣性）

Γ ： Emotional Viscosity Matrix（情動粘性）

K ： Emotional Stiffness Matrix（情動弾性）

Emotional Coefficient Matrix（情動係数行列）

)(tFxkxxm =++ &&& γEquation of Motion（運動方程式）

Equations of Emotion（情動方程式）

EAFKEEEM =+Γ+ &&&

E: Emotion Vector, FEA: Emotional Appraisal

Emotional DynamicsEmotional Dynamics（情動力学）（情動力学）

Time

Em

otio

n V

ecto

r

0

**

*

2 ii

ii

kmγζ =

0det 2 =− MK niω

Emotional AttenuationCoefficient（情動減衰係数）

Emotional Natural AngularFrequency（情動固有周波数）

iTii φφγ Γ=*

iTii Mm φφ=*

iTii Kk φφ=* rEigenvectoi :φ

*

*

i

ini m

k=ω

Stimulus

Visual

Tactile

AuditoryTemperature

Olfactory

SensationLoose Sight of the Target

Discover the TargetDazzling LightTarget is Near

PushedPushed Strongly

StrokedHit

Loud SoundHeat

AlcoholAmmonia

Cigarette SmokeNo Sense

No Stimulus* “→0” means to converge at “0”

Δa ΔcΔp

++++++

++++

+++

-

-

-

--

--

--

-

-

--0

0000000000

00

0

0

00

→0*→0*

Sensing Personality TableSensing Personality Table

11

11

11

10

0=

Eo NeutralEo SurprisedEo HappinessEo SadnessEo AngerEo FearEo Disgust

Ei NeutralEi SurprisedEi HappinessEi SadnessEi AngerEi FearEi Disgust

Eo

ExpressionPersonality

MatrixEi=

Expression PersonalityExpression Personality

Ei: Input Emotion, Eo: Output Emotion

Demos of EYEDemos of EYE--ChanChan

Additional DemosAdditional Demos

More Behaviors Addition of Consciousness

WABIANWABIAN--2 + EYE2 + EYE--ChanChan= KOBIAN= KOBIAN

Emotional ExpressionsEmotional Expressionsof KOBIANof KOBIAN

The EndThe End

Thank you!Thank you!