cse 591: human-aware robotics - arizona state …human-aware robo.cs human modeling human teammate...
TRANSCRIPT
![Page 1: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/1.jpg)
Human-awareRobo.cs
1
CSE 591: Human-aware Robotics
Instructor: Dr. Yu (“Tony”) Zhang
Location & Times: CAVC 359, Tue/Thu, 9:00--10:15 AM Office Hours: BYENG 558, Tue/Thu, 10:30--11:30AM
Oct 6/Nov 1, 2016
This set of slides borrow from various online sources; it is used for educational purposes only.
SlidesadaptedfromPieterAbbeel(UCBerkeley)
![Page 2: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/2.jpg)
Human-awareRobo.cs
2
Modeling of Humans
Behavior model
![Page 3: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/3.jpg)
Human-awareRobo.cs
3
Goal
Modeling of Humans
GoalBehavior model
Ø Goal and intent selection
Goal
![Page 4: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/4.jpg)
Human-awareRobo.cs
4
Goal
Modeling of Humans
Goal
river
GoalBehavior model
• Goal and intent selection Ø Plan selection
![Page 5: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/5.jpg)
Human-awareRobo.cs
5
Goal
Modeling of Humans
Goal
river
GoalBehavior model
• Goal and intent selection • Plan selection (informed by the capabilities, and influenced by mental states and etc.)
![Page 6: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/6.jpg)
Human-awareRobo.cs
6
Modeling of Humans
Behavior model
• Goal and intent selection • Plan selection (informed by the capabilities, and influenced by mental states and etc.)
Ø Goal/planrecogni?onshouldbeinformedbythebehaviormodel
Ø Howshouldwelearnabehaviormodel?
![Page 7: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/7.jpg)
Human-awareRobo.cs
7
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 8: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/8.jpg)
Human-awareRobo.cs
Humanmodeling
Humanteammate
Human-awareplanner
Observa?ons
Humanmodels
Robotmodels
Plangenera6on
8
Modeling of Humans 1. Nopre-specifiedgoals/plans2. Incompleteobserva6ons
![Page 9: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/9.jpg)
Human-awareRobo.cs
9
LearningChallenges
CompleteObserva6ons
ActualObserva6ons
Observa6ons(par6al)withindefinitegaps
Behavior model
1. Nopre-specifiedgoals/plans2. Incomplete&noisyobserva6ons
![Page 10: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/10.jpg)
Human-awareRobo.csCapability
->:denoteanatomicstatechange
{has_water(AG),has_coffee_beans(AG)}->{has_boilling_water(AG),has_coffee_beans(AG)}->{has_boilling_water(AG),has_ground_coffee_beans(AG)}->{has_coffee(AG)}
Westartwithanincompleterepresenta6on
§ DEFINITION(CAPABILITY)–Givenanagent,acapabilityisamapping,whichisanasser.onabouttheprobabilityoftheexistenceofaplaninfewerthanorequaltoTatomicstatechangesthatcanconnectthetwostates.
Par6alstates
has_water(AG)=>has_ground_coffee_beans(AG)has_boiling_water(AG)=>has_coffee(AG)…WhenT=2
WhenT=3…(includingallcapabili?eswhenT=2)has_water(AG)=>has_coffee(AG)
Boundonthegapsbetweenobserva6ons
10
![Page 11: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/11.jpg)
Human-awareRobo.csCapabilityModel
Capabilitymodelencodesallcapabili6esforagivenT
T-gapcapabilitymodel
Synchroniclinks
Diachroniclinks
11
![Page 12: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/12.jpg)
Human-awareRobo.cs
12
CapabilityModel
![Page 13: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/13.jpg)
Human-awareRobo.csCapabilityModel&EncodedCapabili?es
sI=>sE
Acondi6onalprobability(specifiedbyapar6alini6alandeventualstate)
Jointdistribu6onoverT
Acapability:T-gapcapabilitymodel
Acapabilitymodelencodesthefollowingdistribu6ons:
13
![Page 14: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/14.jpg)
Human-awareRobo.csLearningCapabilityModels
§ Learningmodelstructure
Causalrela?onships(diachroniclinks);variablecorrela?ons(synchroniclinks)
§ Learningmodelparameters Condi?onalprobabili?es
Learningfrom(gap-bounded)plantraces
14
![Page 15: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/15.jpg)
Human-awareRobo.cs
15
ParameterLearning
Learningfromincompletetraces
![Page 16: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/16.jpg)
Human-awareRobo.cs
16
ParameterLearning
LearningsamplesApplyBayesianlearning(assumingbetadistribu6ons):
Weassumethatthemaximumnumberofmissingstateobserva6onsbetweenanytwoobserva6onsinthepar6alplantraceisupperboundedbyT
DEFINITION(T-GAPPARTIALPLANTRACE).AT-gappar.alplantraceisapar.alplantraceinwhichallk[1,2…]<=T
![Page 17: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/17.jpg)
Human-awareRobo.cs
17
UsingCapabilityModels
§ Robotcanpredictthehuman’snextac?onoutcomes
Statepredic6on(goalrecogni6on)
Proac6veassistance(toincreasegoalsuccessprobability)
§ Robotcanreasonabouthowlikelyataskcanbeachievedbythehuman
![Page 18: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/18.jpg)
Human-awareRobo.cs
18
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 19: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/19.jpg)
Human-awareRobo.cs
19
Goal
Modeling of Humans
Goal
Goal
![Page 20: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/20.jpg)
Human-awareRobo.cs
20 Safety?Time?Comfort?Wai?ng?me?Speed?
![Page 21: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/21.jpg)
Human-awareRobo.cs
21
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 22: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/22.jpg)
Human-awareRobo.cs
22
Reward:R(s)Decayingfactor:Policy:π
MarkovDecisionProcess
![Page 23: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/23.jpg)
Human-awareRobo.cs
23
![Page 24: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/24.jpg)
Human-awareRobo.cs
24
![Page 25: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/25.jpg)
Human-awareRobo.cs
25
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 26: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/26.jpg)
Human-awareRobo.cs
26
![Page 27: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/27.jpg)
Human-awareRobo.cs
27
![Page 28: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/28.jpg)
Human-awareRobo.cs
28
![Page 29: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/29.jpg)
Human-awareRobo.cs
29
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 30: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/30.jpg)
Human-awareRobo.cs
30
![Page 31: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/31.jpg)
Human-awareRobo.cs
31
![Page 32: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/32.jpg)
Human-awareRobo.cs
32
![Page 33: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/33.jpg)
Human-awareRobo.cs
33
![Page 34: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/34.jpg)
Human-awareRobo.cs
34
![Page 35: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/35.jpg)
Human-awareRobo.cs
35
![Page 36: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/36.jpg)
Human-awareRobo.cs
36
![Page 37: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/37.jpg)
Human-awareRobo.cs
37
![Page 38: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/38.jpg)
Human-awareRobo.cs
38
![Page 39: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/39.jpg)
Human-awareRobo.cs
39
![Page 40: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/40.jpg)
Human-awareRobo.cs
40
![Page 41: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/41.jpg)
Human-awareRobo.cs
41
![Page 42: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/42.jpg)
Human-awareRobo.cs
42
![Page 43: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/43.jpg)
Human-awareRobo.cs
43
![Page 44: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/44.jpg)
Human-awareRobo.cs
44
Directlycomputeapolicy!
![Page 45: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/45.jpg)
Human-awareRobo.cs
45
[Abbeel&Ng,2004]
![Page 46: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/46.jpg)
Human-awareRobo.cs
46
![Page 47: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/47.jpg)
Human-awareRobo.cs
47
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons
![Page 48: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/48.jpg)
Human-awareRobo.cs
48
![Page 49: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/49.jpg)
Human-awareRobo.cs
49
![Page 50: CSE 591: Human-aware Robotics - Arizona State …Human-aware Robo.cs Human modeling Human teammate Human-aware planner Observaons Human models Robot models Plan generaon 8 Modeling](https://reader034.vdocuments.net/reader034/viewer/2022050502/5f94db87814715527310c420/html5/thumbnails/50.jpg)
Human-awareRobo.cs
50
Outline
BehaviormodelingØ Capabilitymodel
GoalpreferenceØ InverseRL
• WhyIRL• InverseRLvs.Behavioralcloning• Mathema?calformula?onofIRL• Applica?ons