airsim: a powerful simulator for robotics research · import cv2 import sys ('127.e.e.1')...
TRANSCRIPT
![Page 1: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/1.jpg)
AirSim: A Powerful Simulator for
Robotics Research
Yulai Weng
CIS 700/002: Topics in Safe Autonomy
Department of Computer and Information Science
School of Engineering and Applied Science
University of Pennsylvania
02/12/2019
1
![Page 2: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/2.jpg)
2
![Page 3: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/3.jpg)
Successes of Machine Intelligence
Real World Flying Systems
![Page 4: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/4.jpg)
Three Fundamental Challenges
4
![Page 5: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/5.jpg)
Old Version of UAV Simulators
5
![Page 6: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/6.jpg)
AirSim: The Next Generation Simulator
6
![Page 7: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/7.jpg)
What Does Simulator Enables?
7
![Page 8: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/8.jpg)
Why Use Unreal Engine?
8
![Page 9: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/9.jpg)
The Heart of the Vehicle
9
![Page 10: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/10.jpg)
How Simulator Works?
10
![Page 11: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/11.jpg)
What Does Physics Engine Do?
11
![Page 12: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/12.jpg)
The Physics Loop
12
![Page 13: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/13.jpg)
May the Force & Torque be with You
13
![Page 14: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/14.jpg)
Physics: Dynamics
14
![Page 15: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/15.jpg)
Simulating IMU
15
![Page 16: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/16.jpg)
Simulating Barometer
16
![Page 17: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/17.jpg)
Simulating Magnetometer
17
![Page 18: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/18.jpg)
The Architecture of AirSim
18
![Page 19: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/19.jpg)
AirSim is Open Source
19
![Page 20: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/20.jpg)
Experiment: Simulation vs. Reality
20
![Page 21: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/21.jpg)
AirSim Has APIs
21
![Page 22: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/22.jpg)
Make Drone Move in AirSim Using APIs
22
![Page 23: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/23.jpg)
23
![Page 24: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/24.jpg)
AirSim Extensibility
24
![Page 25: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/25.jpg)
AgIoT: Precision Agriculture
25
![Page 26: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/26.jpg)
AgIoT: Vision
26
![Page 27: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/27.jpg)
AgIoT: System Architecture
27
![Page 28: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/28.jpg)
Autonomous 3D Scanning of
Large Structures
28
![Page 29: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/29.jpg)
Expands AirSim AI Simulator to
Include Autonomous Car Research
29
![Page 30: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/30.jpg)
End-to-End Deep Learning for
Autonomous Driving
30
![Page 31: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/31.jpg)
Deep Reinforcement Learning
for Autonomous Driving
31
![Page 32: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/32.jpg)
AirSim on Unity
32
![Page 33: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result](https://reader030.vdocuments.net/reader030/viewer/2022041100/5ed88f1a6714ca7f4768268f/html5/thumbnails/33.jpg)
Thanks !
33