obstacle avoidance with kinect

Download Obstacle Avoidance with Kinect

Post on 20-Nov-2015

8 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

By using Kinect sensor, the laser scan is faked for obstacle avoidance. PCL facilities are utilized for this project.

TRANSCRIPT

  • Robot Obstacle Avoidance using the Kinect

    R A S O U L M O J T A H E D Z A D E H

    Master of Science Thesis Stockholm, Sweden 2011

  • Robot Obstacle Avoidance using the Kinect

    R A S O U L M O J T A H E D Z A D E H

    Masters Thesis in Computer Science (30 ECTS credits) at the Systems, Control and Robotics Masters Program Royal Institute of Technology year 2011 Supervisor at CSC was John Folkesson Examiner was Danica Kragic TRITA-CSC-E 2011:107 ISRN-KTH/CSC/E--11/107--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.kth.se/csc

  • Abstract

    Kinect for Xbox 360 is a low-cost controller-free deviceoriginally designed for gaming and entertainment experi-ence by Microsoft Corporation. This device is equippedwith one IR camera, one color camera and one IR projectorto produce images with voxels (depth pixels). This addi-tional dimension to the image, makes it a tempting deviceto use in robotics applications. This work presents a so-lution using the Kinect sensor to cope with one importantaspect of autonomous mobile robotics, obstacle avoidance.

    Modeling the environment based on the point cloud ex-tracted from the depth image data as well as an obstacleavoidance method using the straight line segments and cir-cle arcs, were the main focus of the thesis. The environmentis represented by a set of polygons and the obstacle avoid-ance algorithm attempts to find a collision-free path fromthe current position of the robot to the target pose subjectto the shortest possible path considering a safeguard.

    The whole algorithm was implemented and simulated inMatlab successfully. However, it is partially implementedin C++ and integrated into the current software architec-ture of the project CogX running at CVAP/CAS depart-ment, KTH. A mobile robotic platform based on PioneerP3-DX which was equipped with a LIDAR module whichthe previous obstacle avoidance algorithm relied on. Theshortcoming was not being able to detect all types of ob-stacles. This motivated using the Kinect sensor to extendthe ability to detect obstacles and hence to improve the lo-cal path planner and obstacle avoidance components in thesoftware.

  • Acknowledgements

    I would like to express my gratitude to Dr. John Folkesson, my mainadvisor, for his supervision, great engagement and guidance. I owe many thanksto Dr. Patric Jensfelt for his great effort in directing the program Systems,Control and Robotics during two years of my master studies. I would also liketo thank Alper Aydemir for helping me in implementation part of the thesis.

  • Contents

    1 Introduction 11.1 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

    2 Kinect - A 3D Vision System 32.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

    2.1.1 Depth measurement . . . . . . . . . . . . . . . . . . . . . . . 42.1.2 Lens Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2.1 OpenNI framework . . . . . . . . . . . . . . . . . . . . . . . . 6

    3 Environment Modeling 93.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    3.1.1 Full camera model . . . . . . . . . . . . . . . . . . . . . . . . 103.1.2 Point cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.3 Mapping into 2-dimensional space . . . . . . . . . . . . . . . 143.1.4 Data clustering . . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.5 Concave-hull estimator . . . . . . . . . . . . . . . . . . . . . . 193.1.6 Building a model of the environment . . . . . . . . . . . . . . 21

    4 Obstacle Avoidance and Path Planning 254.1 Configuration Space . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.2 Continuous Geometric Environments . . . . . . . . . . . . . . . . . . 26

    4.2.1 Visibility graph . . . . . . . . . . . . . . . . . . . . . . . . . . 274.2.2 Voronoi diagram . . . . . . . . . . . . . . . . . . . . . . . . . 27

    4.3 Decomposition-based Environments . . . . . . . . . . . . . . . . . . . 284.3.1 Cell decomposition . . . . . . . . . . . . . . . . . . . . . . . . 284.3.2 Occupancy grid mapping . . . . . . . . . . . . . . . . . . . . 29

    4.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.4.1 Finding the vertices . . . . . . . . . . . . . . . . . . . . . . . 304.4.2 Searching for non-colliding path . . . . . . . . . . . . . . . . 324.4.3 Arc joints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.4.4 A safeguard for the path . . . . . . . . . . . . . . . . . . . . . 344.4.5 Obstacle avoidance . . . . . . . . . . . . . . . . . . . . . . . . 36

  • 5 Implementation 395.1 Software architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.3 Rigid body motion of the robot . . . . . . . . . . . . . . . . . . . . . 43

    6 Summary and conclusion 476.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.2 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

    Bibliography 51

  • Chapter 1

    Introduction

    Robotic systems with mobility should be able to explore the environment au-tonomously. Human being, animals and even insects are able to explore theirsurrounding environment safely. They can move around and carry out differenttasks without colliding with static and dynamic obstacles. The main perceptioninvolved in this task is vision when mixed with other kinds of sensing such as touchand hearing makes a perfect input for the biological learning system to learn how tooptimally perform obstacle avoidance in unstructured environments. They silentlycreate a map of the environment, plan a safe path and execute a real-time obstacleavoidance. Adding such capability is an essential requirement for a fully autonomousmobile robot to be able to carry out its tasks.

    As some examples, domestic robots specifically designed to clean a house cannothandle the assigned tasks without an effective obstacle avoidance and path planningsystem as well as mapping the environment. Some of the commercially availabledomestic robots are shown in Figure 1.1.

    The importance of sensing motivated the robotics community to examine differ-ent kinds of sensors mostly ranging from sonars and laser range scanners to stereovision systems. Kinect for Xbox 360 is a low-cost vision device equipped with oneIR camera, one color camera and one IR projector to produce RGB images as wellas voxel (depth-pixel) images. Albeit the main aim of original design by MicrosoftCorporation was for video gaming and entertainment but a great interest in therobotics community to examine their capabilities soon started after the producthad been released. The ability of the device to add one extra dimension to theimages is the key feature which makes it potentially invaluable as a low-cost sensorin robotics systems with wide range of applications.

    The main objectives of this work were to construct a model of the environmentby using the data from the Kinect, given this model, designing a path planner tobuild a non-colliding safe path and finally implement (partially/completely) thealgorithm in C++ on the mobile robot platform Pioneer P3-DX, dedicated to theproject CogX running at CVAP/CAS department, KTH.

    The algorithms developed for both the modeling and obstacle avoidance were

    1

  • CHAPTER 1. INTRODUCTION

    (a) First generationRoomba vacuum

    (b) Home AssistantRobot AR

    (c) daily life mobilerobot assistant Care-O-bot III

    Figure 1.1: Some domestic robots with different level of perception, actuation andintelligence designed to assist people in domestic environments.

    implemented and simulated in Matlab software. The implementation on the robotincluded integration to the current software architecture which utilized LIDAR sen-sor data only to build a 2D grid map as well as a specifically metric map using byobstacle avoidance component. The lack of ability to detect obstacles standing inhigher or lower positions than the LIDAR was a potential danger for the robot tocollide with. Adding the extra information receiving from the Kinect sensor andfuse to the laser data was the main implementation idea performed.

    1.1 OutlineIn chapter 2 an introduction to the Kinect sensor with more details on hardware andsoftware is given. Chapter 3 discusses modeling of the environment based on thedepth data streaming from the Kinect sensor and presents some theory and results.Chapter 4 describes in detail the methodology developed for pre-computation ofa non-colliding path connecting the start and goal poses. Implementation of thealgorithm is discussed in chapter 5. Problems we faced during the project and theconclusion are summarized in chapter 6.

    2

  • Chapter 2

    Kinect - A 3D Vision System

    Microsoft Corporation announced and demonstrated a new add-on device for theXbox 360 video game platform named as Project Natal (which later called as Kinectsensor) in June 2009 which attracted the robotics community to evaluate it as apotentially valuable device. Kinect lunched on November 2010 in north America andEurope, though. The amount of official information published by the manufacturer,Microsoft is less than required for such a research on the device. Hence, the followingcontent of information is mostly derived by sources who obtained the data throughreverse engineering efforts or so-called hacking.

    2.1 Hardware

    Kinect is based on software technology developed internally by Rare [31], a sub-sidiary of Microsof

Recommended

View more >