introduction to mobile robotics welcome - uni-freiburg.de
TRANSCRIPT
![Page 1: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/1.jpg)
1
Wolfram Burgard
Welcome
Introduction toMobile Robotics
![Page 2: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/2.jpg)
Today
This course
Robotics in the past and today
2
![Page 3: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/3.jpg)
3
Organization
Self Studylecture recordings, no on-site lectures
Tue 13:00 – 14:00Q&A session, discussions
Thu 13:00 – 14:30homework, practical exercises (Python)
Web page: http://ais.informatik.uni-freiburg.de/
Exam: Oral or written
![Page 4: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/4.jpg)
People
Teaching:
Wolfram Burgard
Teaching assistants:
Daniel Büscher
Lukas Luft
Johannes Meyer
Shengchao Yan
4
![Page 5: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/5.jpg)
5
Goal of this course
Provide an overview of problems and approaches in mobile robotics
Probabilistic reasoning: Dealing with noisy data
Hands-on experience
![Page 6: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/6.jpg)
Content of this Course
1. Linear Algebra
2. Wheeled Locomotion
3. Sensors
4. Probabilities and Bayes
5. Probabilistic Motion Models
6. Probabilistic Sensor Models
7. Mapping with Known Poses
8. The Kalman Filter
9. The Extended Kalman Filter
10.Discrete Filters
11.The Particle Filter, MCL
6
12. SLAM: Simultaneous Localization and Mapping
13. SLAM: Landmark-based FastSLAM
14. SLAM: Grid-based FastSLAM
15. SLAM: Graph-based SLAM
16. Techniques for 3D Mapping
17. Iterative Closest Points Algorithm
18. Path Planning and Collision Avoidance
19. Multi-Robot Exploration
20. Information-Driven Exploration
21. Summary
![Page 7: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/7.jpg)
Reference Book
Thrun, Burgard, and Fox:“Probabilistic Robotics”
![Page 8: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/8.jpg)
Relevant other Courses
Foundations of Artificial Intelligence
Computer Vision
Machine Learning
and many others from the area of cognitive technical systems.
8
![Page 9: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/9.jpg)
Opportunities
Projects
Practical courses
Seminars
Theses
… your future!
9
![Page 10: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/10.jpg)
Autonomous Robot Systems
perceive their environment and
generate actions to achieve their goals.
environment
sense
act
model
![Page 11: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/11.jpg)
Tasks Addressed that Need to be Solved by Robots
Navigation Perception Learning Cooperation Acting Interaction Robot development Manipulation Grasping Planning Reasoning
…
![Page 12: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/12.jpg)
13
Robotics Yesterday
• Highly repeatable tasks
• Robots bolted to the ground, often caged
• Limited to no perception
• Very little “AI” Picture: Bachmann, Kuka Roboter GmbH
![Page 13: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/13.jpg)
14
Current Trends in Robotics
Robots are (partly) moving away from factory floors …
Entertainment, toys
Personal services
Medical, surgery
Industrial automation
Hazardous environments (mining, harvesting, space, underwater)
Self-driving cars
…
![Page 14: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/14.jpg)
15
Shakey the Robot (1966)
![Page 15: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/15.jpg)
16
Shakey the Robot (1966)
![Page 16: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/16.jpg)
19
The Helpmate System
![Page 17: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/17.jpg)
Autonomous Vacuum Cleaners
![Page 18: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/18.jpg)
Autonomous Lawn Mowers
21
![Page 19: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/19.jpg)
22
DARPA Grand Challenge
[Courtesy by Sebastian Thrun]
![Page 20: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/20.jpg)
Walking Robots
[Courtesy by Boston Dynamics]
![Page 21: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/21.jpg)
Driving in the Waymo Car
JJ Ricks in the Waymo Car #47
![Page 22: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/22.jpg)
Folding Towels
![Page 23: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/23.jpg)
32
Rhino (Univ. Bonn + CMU, 1997)
![Page 24: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/24.jpg)
33
Minerva (CMU + Univ. Bonn, 1998)
Minerva
![Page 25: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/25.jpg)
Robotics in Freiburg
34
![Page 26: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/26.jpg)
Autonomous Parking
![Page 27: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/27.jpg)
Autonomous QuadrotorNavigation
Custom-built system:
laser range finder
inertial measurement unit
embedded CPU
laser mirror
![Page 28: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/28.jpg)
Precise Localization and Positioning for Mobile Robots
![Page 29: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/29.jpg)
Obelix – A Robot Traveling to Downtown Freiburg
![Page 30: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/30.jpg)
The Obelix Challenge (Aug 21, 2012)
![Page 31: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/31.jpg)
The Tagesthemen-Report
![Page 32: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/32.jpg)
Brain-controlled Robots
41
![Page 33: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/33.jpg)
Teaching: Student Project onthe Autonomous Portrait Robot
![Page 34: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/34.jpg)
Final Result
![Page 35: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/35.jpg)
Other Cool Stuff from AIS
44
![Page 36: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/36.jpg)
Accurate Localization
KUKA omniMove (11t)
Safety scanners
Error in the area of millimeters
Even in dynamic environments
![Page 37: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/37.jpg)
Fuselage assembly
20 vehicles to transport industrial robots for drilling and filling of 60,000 fasteners in
6 vehicles for logistics of parts, work stands and fuselages
26 Units installed at Boeing
![Page 38: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/38.jpg)
![Page 39: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/39.jpg)
Learning User Preferences
Task preferences are subjective
Fixed rules do not match all users
Constantly querying humans is suboptimal
How to handle new objects?
Where does this
go?
![Page 40: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/40.jpg)
?
Collaborative Filtering
•…
•-
•-
•-
![Page 41: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/41.jpg)
Collaborative Filtering
•…
•-
•-
•-
![Page 42: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/42.jpg)
Online Prediction of Preferences
![Page 43: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/43.jpg)
Deep Learning Applications
RGB-D object
recognition
Images human part
segmentation
Sound terrain
classification
![Page 44: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/44.jpg)
Fusion layers automatically learn to combine feature responses of the two network streams
During training, weights in first layers stay fixed
DCN for Object Recognition
![Page 45: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/45.jpg)
Learning Results
Method RGB Depth RGB-D
CNN-RNN 80.8 78.9 86.8
HMP 82.4 81.2 87.5
CaRFs N/A N/A 88.1
CNN Features 83.1 N/A 89.4
This work, Fus-CNN 84.1 83.8 91.3
•Category-Level Recognition [%] (51 categories)
•[Lai et. al, 2011]
![Page 46: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/46.jpg)
Network Architecture
Fully convolutional network
Contraction and expansion of network input
Up-convolution operation for expansion
Pixel input, pixel output
![Page 47: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/47.jpg)
Deep Learning for Body Part Segmentation
•Input Image •Ground Truth •Segmentation mask
![Page 48: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/48.jpg)
Deep Learning for Terrain Classification using Sound
![Page 49: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/49.jpg)
Network Architecture
Novel architecture designed for unstructured sound data
Global pooling gathers statistics of learned features across time
![Page 50: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/50.jpg)
Data Collection
Asphalt
Wood
Offroad
CobbleStone
PavingGrassMowed Grass
CarpetLinoleum
P3-DX
![Page 51: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/51.jpg)
Results - Baseline Comparison
16.9% improvement over the previous state of the art 99.41% using a 500ms window
(300ms window)
[1]
[2]
[3]
[4]
[5]
[6]
[1] T. Giannakopoulos, K. Dimitrios, A. Andreas, and T. Sergios, SETN 2006
[2] M. C. Wellman, N. Srour, and D. B. Hillis, SPIE 1997.
[3] J. Libby and A. Stentz, ICRA 2012
[4] D. Ellis, ISMIR 2007
[5] G. Tzanetakis and P. Cook, IEEE TASLP 2002
[6] V. Brijesh , and M. Blumenstein, Pattern Recognition Technologies and Applications 2008
![Page 52: Introduction to Mobile Robotics Welcome - uni-freiburg.de](https://reader031.vdocuments.net/reader031/viewer/2022012104/616a1a4f11a7b741a34ed4a5/html5/thumbnails/52.jpg)
Thank you
… and enjoy the course!
68