a gui design of robot motion and task planning based on linear …1249798/... · 2018. 9. 20. ·...

IN DEGREE PROJECT ELECTRICAL ENGINEERING,SECOND CYCLE, 30 CREDITS

,

A GUI Design of Robot Motion and

Task Planning Based on Linear

Temporal Logic

TENGFAN LIN

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

A GUI Design of Robot Motionand Task Planning Based onLinear Temporal Logic

TENGFAN LIN

Systems, Control and RoboticsDate: August 28, 2018Supervisor: Meng Guo, Lars LindemannExaminer: Dimos DimarogonasSchool of Electrical Engineering and Computer Science

iii

Abstract

Recent approaches solve the problem of robot motion and task plan-ning by using formal methods-based model checking algorithms. Inthis work, we consider the software package P-MAS-TG, an automatictool to generate correct-by-design controllers for robot motion and taskplanning. The robot motion can be modelled as a finite-state transitionsystem and the task can be represented by a linear temporal logic for-mula. Then, using a model checking algorithm, an accepting path forthe robot motion can be found so that the robot satisfies the specificlinear temporal logic task. In this thesis, the process of searching forthe accepting path of the robot motion by P-MAS-TG is visualized bymeans of a graphical user interface (GUI) in form of an rqt plug-inwhere all requested inputs for the P-MAS-TG package can be definedin the GUI so that the process is simplified and user-friendly. More-over, the GUI provides a simple hybrid control mechanism betweenthe initial linear temporal logic task and a temporary task that is in-serted by a human.

All functions of the GUI are implemented and demonstrated onthe TurtleBot platform. The experiment shows that the GUI and theP-MAS-TG are successfully integrated. Features of the P-MAS-TG areperformed with the GUI to conduct motion planning and task plan-ning on the TurtleBot.

iv

Sammanfattning

Nya metoder lösa problemet med robot motion och task planningmed hjälp av formella metoder-baserade modellkontroll algoritmer. Idetta exjobb anser vi att programvarupaketet P-MAS-TG, ett automa-tiskt verktyg för att generera correct-by-design controllers för robotmotion och task planning. Robot motion kan modelleras som ett änd-ligt tillstånd övergångssystem och uppgiften kan representeras av enlinjär temporal logik formel. Sedan, med hjälp av en modellkontrollalgoritm, kan en accepterande väg för roboten rörelse hittas så att ro-boten uppfyller de specifika linjära temporala logik uppgift. I dennauppsats processen att söka efter den accepterande sökvägen till robotmotion av P-MAS-TG visualiseras med hjälp av ett grafiskt användar-gränssnitt (GUI) i form av en rqt plug-in där alla efterfrågade indataför P-MAS-TG-paketet kan definieras i GUI så att process är förenklatoch användarvänligt. Dessutom ger GUI en enkel hybrid kontrollme-kanism mellan den ursprungliga linjära temporala logisk uppgift ochen tillfällig uppgift som infogas av en människa.

Alla funktioner i GUI implementeras och demonstreras på TurtleBot-plattformen. Experimentet visar att GUI och P-MAS-TG är framgångs-rikt integrerade. Egenskaper hos P-MAS-TG utförs med GUI för attgenomföra motion planning och task planning på TurtleBot.

Contents

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Background 72.1 Motion Planning . . . . . . . . . . . . . . . . . . . . . . . 72.2 Task Planning . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Model Checking . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Method 113.1 Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Finite-State Transition System . . . . . . . . . . . . 113.1.2 Linear Temporal Logic . . . . . . . . . . . . . . . . 133.1.3 LTL Model Checking . . . . . . . . . . . . . . . . . 13

3.2 Robot Motion Modelling . . . . . . . . . . . . . . . . . . . 153.3 Task Specification as LTL formula . . . . . . . . . . . . . . 153.4 Task planning . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.4.1 Single Task Plan Synthesis . . . . . . . . . . . . . . 163.4.2 Task Planning with Human Intervention . . . . . 17

4 Software Implementation 204.1 Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1.1 Robot Operating System . . . . . . . . . . . . . . . 204.1.2 Software Package P-MAS-TG . . . . . . . . . . . . 224.1.3 Rqt . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2 LTL planner plug-in in rqt . . . . . . . . . . . . . . . . . . 254.2.1 Map definition . . . . . . . . . . . . . . . . . . . . 254.2.2 Plan synthesis . . . . . . . . . . . . . . . . . . . . . 274.2.3 Navigation . . . . . . . . . . . . . . . . . . . . . . . 31

v

vi CONTENTS

5 Simulation and Result 325.1 TurtleBot and Gazebo . . . . . . . . . . . . . . . . . . . . . 335.2 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . 345.3 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5.3.1 Result of Simulation 1 . . . . . . . . . . . . . . . . 355.3.2 Result of Simulation 2 . . . . . . . . . . . . . . . . 375.3.3 Result Analysis . . . . . . . . . . . . . . . . . . . . 38

6 Experiment and Result 406.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . 40

6.1.1 Setup of Experiment 1 . . . . . . . . . . . . . . . . 416.1.2 Setup of Experiment 2 . . . . . . . . . . . . . . . . 41

6.2 Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.2.1 Result of Experiment 1 . . . . . . . . . . . . . . . . 416.2.2 Result of Experiment 2 . . . . . . . . . . . . . . . . 426.2.3 Result Analysis . . . . . . . . . . . . . . . . . . . . 44

7 Conclusion and Future Work 457.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 45

7.2.1 Full Features of P-MAS-TG . . . . . . . . . . . . . 467.2.2 Multi-agent Collaboration . . . . . . . . . . . . . . 46

Bibliography 47

A User Manual 52A.1 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 52

A.1.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . 52A.1.2 Installation . . . . . . . . . . . . . . . . . . . . . . . 52A.1.3 Execution . . . . . . . . . . . . . . . . . . . . . . . 52

A.2 Map Definition . . . . . . . . . . . . . . . . . . . . . . . . 53A.3 Plan Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . 53A.4 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 53A.5 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Chapter 1

Introduction

As soon as the word Robot was introduced by Czech writer Karel Capekin 1920, it never fails to fascinate scientists and engineers. Accordingto the writer’s explanation, Robot originates from Robota in Czech andis widely known as "work" or "labour" in general. This concept pro-vides possibilities that human-beings could be released from the heavyphysical work and live a better life. Around this fascinating topic, sci-entists and engineers are engaged in inventing or improving relevanttheories and technologies with enormous passion.

Figure 1.1: HAL 9000 in 2001: A Space Odyssey

1

2 CHAPTER 1. INTRODUCTION

Also in the artistic field, many artists have also endeavoured todiscover a deeper comprehension of robot. The most famous workwould be the Three Laws of Robotics created by Isaac Asimov. In addi-tion, we could also see more and more robots with artificial generalintelligence(AGI) showing in films, such as HAL 9000 in 2001: A SpaceOdyssey.

With constrains of technology, we can still not develope an AGI torealize functions as shown in the film, despite hundred years havingpassed. The AGI looks still like a dream, but the weak AI has beenimplemented and widely used both in the industry and our daily life.Nowadays we have discovered a huge amount of ways to establishan autonomous robot system to substitute humans’ role on physicallabour.

Figure 1.2: Autonomous cars from google, retrieved fromhttps://www.androidheadlines.com/

The autonomous car, which is also known as the self-driving car,is a typical autonomous robot system drawing the most significant at-traction in recent years and represents the hottest trend on both aca-demic and industrial field. The autonomous car combines a variety oftechniques to perceive their surroundings, e.g radar, laser light, GPS,odometry and computer vision, and it is capable of sensing its envi-ronment and navigating without human input [43]. For example, theautonomous car from google, as shown in figure 1.2, has already rolledout over 25 cities [30]. Moreover, not only the company from SiliconValley(Waymo, Tesla, Uber etc.) but also traditional vehicle companiesare enrolled in the technical field. Now companies like Volvo, VW and

CHAPTER 1. INTRODUCTION 3

so on are all developing their own self-driving system.

Figure 1.3: UAV for Amazon Prime Air, retrieved fromhttps://www.popularmechanics.com/

The unmanned aerial vehicle(UAV) is another typical example thathas been already deployed in our daily life. Similar to the autonomouscar, the UAV also synthesises different kinds of sensors such as GPS,IMU and camera to perceive environment, and reacts stability whendisturbances are involved. The UAV has already been launched onthe business market by Amazon in 2016. Amazon revealed its planfor Amazon Prime Air based on the UAV technology to fly individualpackages autonomously to customers within 30 minutes of ordering[42].

Figure 1.4: DJI Inspire 2, retrieved from https://www.indiamart.com/

In the individual consumer market, DJI is the most famous com-pany that has an entire product line from professional drone to minidrone. Their products are changing our life. Now consumers can evenrecord an aerial video by themselves, more and more movie-like vlogshaven been shown on the Youtube.


1.1 Motivation

All robot systems that mentioned above are autonomous. Instead ofautomated system, autonomous system relies on as minimal artificialaids as possible. Robot systems are authorized to handle multiplecomplicated tasks and situations automatically without human inter-ventions, or even cooperate with other robots.

Figure 1.5: A example of work flow for UAVs. The flow is easy to exe-cute but only based on locations, more complex tasks are not involved,retrieved from https://www.microdrones.com/

In general, the mentioned robot systems can perfectly process a sin-gle task like driving from A to B and totally be autonomous when taskis running. But in practice the desired task could be much more com-plex and normally presented as series of tasks. In the case of the UAV,they could have a sequence of customers to send or catch packages,and also need to avoid flying-forbidden zones. No matter the tasks areglobal or local, most robot systems still need human inputs to achieve

CHAPTER 1. INTRODUCTION 5

or modify their goals to finish them, which can be considered as a par-tially autonomous system.

With the analysis of series of tasks, it is easy to make a conclusionthat tasks can be arranged in linear temporal sequence, and consists oflocations and actions. In order to fulfil linear temporal tasks, a higher-level planner considering both locations and actions at the same timeneeds to be developed and merged with the point-to-point navigationtechnique [13].

Thanks to the brilliant research by Dimos Dimarogona and MengGuo [14], we have a great promotion on settling down linear tempo-ral task problem. In [13], Meng reorganised the problem in 5 aspects,from the lowest-level of robot motion and task planning scheme to thehighest-level of cooperation of robots with on-line planning and real-time updates of the workspace model.

Figure 1.6: Reorganization of linear temporal tasks problem by Meng,retrieved from [13]. In this thesis we will concentrate on visualizingP1.

In this thesis, we are going to design a graphic user interface (GUI)for robot motion and task planning based on Meng’s work, trying tovisualize the process and provide a better user experience. Due tothe variety of map properties and robot actions under different sce-narios, only single robot in a static fully-known workspace withoutextra properties is considered. Actions are neglected as well, but theAPI is preserved and can be added in the future to adapt to differentdemands.


1.2 Outline

The thesis consists of seven chapters. Chapter 1 is an introduction ofthe thesis, including recent autonomous robot systems and the moti-vation of the thesis; Chapter 2 shows the relative research backgroundbased on the motivation, including robot motion planning, task plan-ning and model checking. Theoretical details of motion planning withthe finite-state transition system and task planning with LTL modelchecking are presented in Chapter 3. Chapter 4 describes the soft-ware implementation of visualizing the P-MAS-TG package with rqton ROS. In Chapter 5, two simulations are conducted to verify the val-idation of the implementation in Chapter 4 and Chapter 6 shows thedeployment of the plug-in on the real robot to conduct experimentssimilar to the simulations in Chapter 5. Chapter 7 is the conclusion ofthe thesis with some potential improvements.

Chapter 2

Background

In this chapter, a brief introduction to the relative research backgroundof the thesis is involved, including robot motion planning, task plan-ning and model checking.

2.1 Motion Planning

Robot motion planning can refer to that looking for a sequence of ac-tions to achieve some specific goals in the continuous workspace, e.gmoving from the initial position to the goal position, or running someoperations like grasping or releasing an object. The basic motion plan-ning problem can be defined as following [20]:

Given an initial position and orientation and a goal position andorientation of a robot in workspace, generate a path specifying a con-tinuous sequence of positions and orientations of the robot avoidingcontact with the obstacles starting at the initial position and orienta-tion, and terminating at the goal position and orientation. Report fail-ure if no such path exists.

Many efficient motion planning algorithms show their abilities onsolving the problem. In [13], the potential-field-based control algo-rithm [22] is applied. Other planning techniques like probabilisticroadmap(PRM) [16] and rapidly-exploring random trees(RRT) [21] areall widely used. In this thesis, for simplification the motion planningproblem is simplified as a point-to-point navigation problem and im-plemented with the navigation package in ROS.

7

8 CHAPTER 2. BACKGROUND

2.2 Task Planning

Motion planning gives solutions to a single kind of task, e.g the robotmoves from an initial position to a goal position, or changes from aninitial pose to a desired pose. When tasks are more complex, it be-comes more complicated to find a motion plan because there are ahuge number of robot states that need to be considered. Treating arobot’s operations only in terms of results on the objects of manipula-tion seems to be one way of solving this problem [26]. Task planningis focused on solving that problem. For example, it is easier for usersto tell a drone "fly to Bob’s home to catch the package and send it toMary" than define a sequence of motions that move from one positionto another and reach a specific pose at the desired position.

Figure 2.1: Schematic view of task planning.

In [13], a task planner considers a robot system as a discrete finite-state transition system. As a result, the task planning problem can be

CHAPTER 2. BACKGROUND 9

reorganized as finding a path that transforms the system from the ini-tial state to the goal state with the given task specification. Particularly,in this thesis the task specification is defined as a linear temporal logicstatement and the robot system is modelled as a finite-state transformsystem(FTS). The basic work flow of task planning is described in fig-ure 2.1.

2.3 Model Checking

Model checking is a technique for automatically verifying correctnessproperties of a finite-state transform system. Given a model of a sys-tem, model checking can exhaustively and automatically check whetherthis model meets a given specification. The basic work flow of modelchecking verification is described in figure 2.2.

Figure 2.2: Schematic view of the model-checking approach [2].

The system can be either a software or a hardware system. Thespecification normally contains safety requirements such as the ab-sence of deadlocks and similar critical states that can cause the sys-tem to crash. Linear temporal logic(LTL) provides a concise and for-mal way to specify both propositional and temporal requirements onthe system behaviour [13]. The model checker examines all relevantsystem states to check whether they satisfy the desired property. Ifall possible system behaviours satisfy the property specification, the

10 CHAPTER 2. BACKGROUND

model checker returns satisfied, otherwise violated is returned and acounterexample is given indicating that an undesired state is reachedin the system [2].

In this thesis, the model-checking algorithm is used for synthesisrather than verification. The purpose of model checking verificationis finding any system state that violates the property and modifyingthe system to achieve the desired specification, while the purpose ofmodel checking synthesis is finding a sequence of system states thatsatisfy the property and mostly optimized regarding a certain costfunction .

Chapter 3

Method

The theoretic background of an automated robot motion and task plan-ning framework for a mobile robot is introduced in this chapter. A fewpreliminaries about the automata theory, basic concepts in LTL andthe model-checking synthesis algorithms are introduced in the firstsection. In the second section, a finite-state transition system is con-structed as the abstract of the robot motion in the workspace. Then thetechnique of how complex tasks can be represented as LTL formulasis shown in the third section. The last section describes the algorithmof finding a plan that satisfies task specifications through the model-checking method and the algorithm of switching tasks between theinitial task and the temporary task inserted by humans.

3.1 Preliminary

3.1.1 Finite-State Transition System

A finite-state transition system is a mathematical description of thebehaviour of systems, plants, controllers or environments with finiteinputs, outputs and internal states and transitions between the states.

Definition 3.1.1. Transition SystemA transition system TS is a tuple (S,Act,→, I, AP, L) where

• S is a set of states

• Act is a set of actions

• →⊆ S × Act× S is a transition relation

11

12 CHAPTER 3. METHOD

• I ⊆ S is a set of initial states

• AP is a set of atomic propositions

• L : S → 2AP is a labelling function

TS is called finite if S,Act, AP are finite.

Here a proposition is a statement that can be either true or false, butnot both. An atomic proposition is one whose truth or falsity does notdepend on the truth or falsity of any other proposition. 2AP denotesthe power set of AP . The labelling function relates a set L(s) ∈ 2AP ofatomic propositions to any state s.

Atomic propositions in the transition system depend on the prop-erties under consideration. A simple choice is to let the state names actas atomic propositions [2].

Definition 3.1.2. Non-deterministic Büchi Automaton(NBA)A Non-deterministic Büchi Automaton(NBA)A is a tupleA = (S, 2AP , δ, S0, F )

where

• S is a finite set of states

• 2AP is an alphabet

• δ : S × 2AP → 2S is a transition function

• S0 ⊆ S is a set of initial states

• F ⊆ S is a set of accepted states

Here the transition function δ can be identified with the relationrelation→⊆ S × 2AP × S given by (q, A, q′) ∈→ iff q′ ∈ δ(q, A) whereA ∈ 2AP .

Given an infinite string ω = A1A2... ∈ (2AP )ω (here (2AP )ω denotesthe set of words that arise from the infinite concatenation of words in2AP ), a run rω for ω in A is an infinite sequence r = q0q1q2..., whereq0 ∈ S0 and qk ∈ δ(qk−1, Ai) for i, k > 0 and Ai ∈ 2AP . rω is acceptingif qi ∈ F for infinitely many i. Correspondingly, ω is accepted by A ifthere is an accepting run of ω in A. Denote Lω(A) as a set of infinitestrings accepted by A.

Notice thatA is non-deterministic, there may exist multiple runs ofthe same word.

CHAPTER 3. METHOD 13

3.1.2 Linear Temporal Logic

Linear temporal logic(LTL) is a formal language for describing futurebehaviours of systems. It provides particularly useful operators forconstructing linear-time properties without explicitly specifying sets,e.g. infinite sequences of subsets of atomic propositions. LTL givesa concise and formal way to specify both propositional and temporalconstraints on the system behaviour [13].

The syntax and semantics of LTL are described in details in in Chap-ter 5 in [2]. Here we only focus on an important theorem talking aboutthe relation between NBA and LTL formula.

Theorem 1. NBA for LTL FormulaThere exists an algorithm that takes an LTL formula φ and returns an NBAA such that Words(φ) = Lω(A).

The proof can be seen at Theorem 5.37 in [2].

Definition 3.1.3. Product of Transition System and NBAGiven a finite-state transition system without terminal state TS = (S,Act,→, I, AP, L) and a non-blocking NBA A = (Q, 2AP , δ, Q0, F ), the productof them TS ⊗A is the following transition system

TS ⊗A = (S ×Q,Act,→′, I ′, AP ′, L′) (3.1)

where

• ∀s, t ∈ S,∀p, q ∈ Q,α ∈ Act and (s, α, t) ∈→, p ∈ δ(q, L(t)), thenthere exists (〈s, q〉, α, 〈t, p〉) ∈→′

• I ′ = {〈s0, q〉 : s0 ∈ I,∃q0 ∈ Q0 and q0 ∈ δ(q0, L(s0))}

• AP ′ = Q,L′ : S ×Q→ 2Q, L′(〈s, q〉) = {q}

3.1.3 LTL Model Checking

In the previous section we have defined finite-state transition systemand LTL formula. The model checking of verification problem is for-mulated as:

• Given: a finite-state transition system TS and an LTL formula φ

• Problem: Does TS satisfies φ?


In order to solve the problem, theorem 2 is introduced.

Theorem 2. Verification of LTL formulaGiven a finite-state transition system without terminal state TS = (S,Act,→, I, AP, L) and a non-blocking NBA Aφ = (Q, 2AP , δ, Q0, F ) based on anLTL formula φ, the following statements are equivalent:

• TS |= φ

• TS ⊗A¬φ |= 32¬F

The proof can be seen at Theorem 4.63 in [2].[40] provides an automata-based approach to present the verifica-

tion process. To check whether an LTL formula φ holds for a finite-statetransition system TS, first the negation NBA A¬φof the input φ needsto be constructed. A¬φ normally represents the bad behaviours of TS.Then compute the product ProTS⊗A¬φ

= TS ⊗A¬φ. Last but not least,if ∃π ∈ Paths(ProTS⊗A¬φ

) that π |= A¬φ, then TS 6|= φ and return anexpressive prefix of φ, otherwise TS |= φ. Algorithm 1 from [2] showsdetails.

Algorithm 1: Automaton-based LTL model-checking for verifica-tion

Result: yes if TS |= φ; otherwise return no and a counterexampleRead finite-state transition system TS and LTL formula φConstruct an NBA A¬φConstruct the product ProTS⊗A¬φ

= TS ⊗A¬φwhile true do

if ∃π ∈ Paths(ProTS⊗A¬φ), π |= A¬φ then

return no && Prefix(π)

elsereturn yes

endend

It is hard to execute the checking process manually. Fortunatelythere have already been off-the-shelf model checkers, e.g. SPIN. In thisthesis, a model checker is included in the software package P-MAS-TG.


3.2 Robot Motion Modelling

The robot motion can be abstracted as a finite-state transition systemas follows: the finite-state transition system for the robot motion is atuple TR = (S,Act,→, I, AP, L), where

• S is a set of robot states, referring to points on the map

• Act is a set of signals generated by the navigation controller, usu-ally provided by the lower controller on the robot

• →R⊆ S × Act × S is a transition relation, referring to the point-to-point movement of the robot

• I ⊆ S is a set of initial states, indicating the start point of therobot

• AP is a set of atomic propositions

• L : S → 2AP is a labelling function defined in definition 3.1 in[13]

For simplicity, denote si →R sj as equivalent as (si, u(t), sj) ∈→R.

3.3 Task Specification as LTL formula

LTL formulas can be used to specify various robot control tasks likesafety, ordering, process and surveillance.

Example 3.3.1. Some examples of Task Specification written as LTLformulas:

• Safety: 2¬φ, globally avoid φ

• Ordering: 3(φ1 ∧3(φ2 ∧3φ3)), visit φ1, φ2, φ3 in order

• Process: 2(φ1 → 3φ2), φ1 requests a response φ2

• Surveillance: 23φ, always visit φ

A sequence of sub-workspace can be specified as an LTL formula,with task planning the robot can drive exactly as specified when thereis an available plan.


3.4 Task planning

In this section, two scenarios are under consideration. The first sce-nario is the plan synthesis for a single task, and another is similar to thesingle task scenario, but with human intervention involved. In prac-tice robots usually run not only initial tasks, but also run temporarytasks inserted by humans. At that time, the initial task is paused andcontinues to be executed after the temporary task has been finished.

3.4.1 Single Task Plan Synthesis

Instead of verification, a single task planning problem is a synthesisproblem in the model checking algorithm. The most significant differ-ence between verification and synthesis is that, in verification we focuson the bad behaviour of the system and return a counterexample if thesystem does not satisfy specifications, while in synthesis we focus onthe good behaviour of the system and return an accepting plan thatsatisfies specifications.

Thus, consider Algorithm 1, a slight modification is needed to dealwith the synthesis problem.

Algorithm 2: Automaton-based LTL model-checking for synthe-sis

Result: no if TS 6|= φ; otherwise return yes and an accepting planRead finite-state transition system TS and LTL formula φConstruct an NBA AφConstruct the product ProTS⊗Aφ

= TS ⊗Aφwhile true do

if ∃π ∈ Paths(ProTS⊗Aφ), π |= Aφ then

return yes && Prefix(π)

elsereturn no

endend

In Algorithm 2, the negation NBA of LTL formula is unnecessary,since we need a plan for good behaviour.

Another problem is the structure of the plan. When a plan π ∈TS ⊗ Aφ is found, it is an infinite set of states and there is no finalstate. For real robot movement, the robot stays in a region eventuallyafter the execution of the plan or repeat a plan sequence under the


surveillance scenario, that means the plan should have the followingstructure,

π = 〈πpre, πsuf〉 = πpre[πsuf ]ω (3.2)

where the prefix πpre is executed only once and the suffix πsuf is re-peated infinitely. This problem is solved in software package P-MAS-TG with weighted finite-state transition system and optimal searchtechnique. The theoretical basis can be approached in Chapter 3 in[13].

3.4.2 Task Planning with Human Intervention

This scenario is much more general than the single task plan synthesis.Few robots run only a single task. In order to deal with a temporarytask requested by human, the robot needs to pause running the gen-eral task, execute the plan synthesis for the temporary task, finish thetemporary task and continue to run the general task. For convenience,the process can be modelled as a hybrid automaton as follows:

the hybrid automaton for task planning with human interventionis a collection H = (Q,X, f, Init,D,E,G,R), where

• Q is a set of discrete states

• X is a set of continuous states

• f : Q×X → X is a vector field. Referring (q, x) ∈ Q×X , f(q0, x)means the robot dynamic when the task q0 is executed

• Init ⊂ Q×X is a set of initial states

• D : Q→ 2x is a domain.

• E :⊂ Q×Q is a set of edges

• G : E → 2x is a guard condition

• R : E ×X → 2x is a reset map

Denote (q, x) ∈ Q×X , as shown above, the hybrid automaton canbe explained in details as follows:


Figure 3.1: The hybrid automaton of task planning with human inter-vention.

• Q = {q0, q1}, where q0 = initial_task, q1 = temporary_task. Thatmeans the robot executes the task switching between the initialone and the temporary one.

• X represents the robot dynamics. It is given in section 3.2 andcan be denoted as x(t) = g(x(t), u(t)), where u(t) is a navigationcontroller to drive the robot for navigation.

• f(q0, x) = g(x(t), uq0(t)) and f(q1, x) = g(x(t), uq1(t)). That de-scribes the robot dynamic under different tasks. For example,f(q0, x) means when q = q0, the robot dynamic is governed byg(x(t), uq0(t)), uq0(t) means the robot is navigated following thetask q0.

• Init = Q×X means the initial state can be any state in the wholestate space.

• D(q0) = {x ∈ R, u(t) = uq0(t)} and D(q1) = {x ∈ R, u(t) =

uq1(t)}.

• E = {(q0, q1), (q1, q0)} means possible transitions are q0 → q1 orq1 → q0.


• G(q0, q1) = {x ∈ R, u(t) = uq1(t)} and G(q1, q0) = {x ∈ R, u(t) =uq0(t)}. That means when the navigation control signal has beenchanged, task follows the change.

• R(q0, q1, x) = R(q1, q0, x) = x. After the change, the continuousstate gets reset.

As shown in figure 3.1, the key of switching the task is setting upthe navigation controller signal. With linking it to the human oper-ation, the robot can run the temporary task called by the human op-erator. Algorithm 3 shows the process of dealing with the temporarytask.

Algorithm 3: Task planning switch between the original task andthe temporary task

Result: execute q1 if q1 6= ∅ until being terminated or completed,otherwise execute q0

while true doRead initial task q = q0Plan synthesis p for q(u = uq0)while execute p do

Read temporary task q1if q1 6= ∅ then

q = q1replan p for q(u = uq1)if p terminated or completed then

q1 = ∅break

endend

endend

Chapter 4

Software Implementation

The software implementation of mentioned theories and algorithmsin Chapter 3 has already been accomplished by Meng in the softwarepackage P-MAS-TG and can be approached from [24]. This chapterpresents the graphic user interface(GUI) implementation of P-MAS-TG. We start with the environment of the implementation, includinga brief introduction of ROS, P-MAS-TG and rqt, which is the platformwe rely on to design the GUI. Then we link the GUI to P-MAS-TG andthe whole structure and logic behind the GUI are presented in the restsections.

4.1 Platform

The thesis work is implemented on ROS indigo on Ubuntu 14.04. Afull desktop distribution of ROS with turtlebot package is requested.P-MAS-TG package from Meng is also essential and its preliminarycan be found in [24]. The GUI is implemented with PyQt4 integratedin rqt and some features from matplotlib are also involved.

4.1.1 Robot Operating System

The Robot Operating System (ROS) is a flexible framework for imple-menting robot software. It is a collection of tools, libraries, and con-ventions that aims at simplifying the task of creating complex and ro-bust robot behavior across a wide variety of robotic platforms [32]. Inrecent years ROS becomes more and more popular due to its conve-nience on organizing complex tasks. It provides an uniform interface

20

CHAPTER 4. SOFTWARE IMPLEMENTATION 21

for different functional packages. Researchers and engineers can getrid of implementing communication protocol for different robots andfocus on their technical interests such as mapping and navigation etc.

Figure 4.1: Robot operating system(ROS)

Figure 4.1 describes the basic mechanism of ROS. In order to un-derstand figure 4.1, a few fundamental concepts should be introducedfirst.

NodeA node is a process that performs computation. It can reach the hard-ware like sensors, motors, and monitoring systems. A robot controlsystem usually comprises many nodes. It is an executable file thatuses ROS client library to communicate with other nodes through Top-ics and Services [17].

TopicTopics are named buses over which nodes exchange data. In general,nodes exchange data in anonymous mode, they do not know whichnode they exchange data with. Nodes publish messages to or subscribemessages from topics. There can be multiple publishers and subscribersto a topic [9].

ServiceROS also provides service as another method of communication be-tween nodes. Topics are used for real-time data exchange, while serviceis appropriate for long-term requests and responses. For example, in

22 CHAPTER 4. SOFTWARE IMPLEMENTATION

this thesis the discrete plan performs as a sequence of tasks. We set thesequence as a service to broadcast the next task. It is only called whenthe previous task has been done [18].

MessageNodes communicate with each other by publishing messages to topics.A message is a simple data structure, comprising typed fields. It spec-ifies the structure of the data generated or received from Nodes. Forexample, in this thesis the discrete plan is considered as a string mes-sage [12].

Following the mentioned basic concepts, the running mechanismof ROS can be descried as in figure 4.1. All nodes are running inde-pendently and simultaneously. They can publish different formats ofmessages in a specific topic, and can subscribe messages from differenttopics at the same time. When synchronous communication betweentwo nodes is needed, nodes call service to exchange messages.

Figure 4.2: Architecture of P-MAS-TG.

4.1.2 Software Package P-MAS-TG

The full name of P-MAS-TG implemented by Meng in [13] is plannerfor multi-agent systems with temporal goals. It features both the motion


planning and task planning modules described in Chapter 2 and 3.The architecture of the package is shown in figure 4.2.

Recall the robot motion model TR = (S,Act,→, I, AP, L) we con-structed in section 3.2. As can be seen in figure 4.2, the input file, in-cluding AP with L(names of regions),→ with Act(possible transitionbetween regions) and I(starting point) are used to construct a finite-state transition system as a NetworkX graph structure [15]. The FTSupdates(label 5) through reading data from sensors and communica-tions. An LTL formula is also read as the task specification and trans-ferred to an NBA(label 1, Theorem 1), the soft NBA is not consideredin this thesis. Then we have the product automaton (label 3, Definition3.1.3) and it is updated when the FTS has been updated (label 6). Afterplan synthesis(label 4, Algorithm 2), we have got an initial plan.

Figure 4.3: Simplified work flow of P-MAS-TG.

In this thesis, we will not use all functions in the package. The sim-plified work flow can be seen in figure 4.3. Main differences betweenfigure 4.2 and figure 4.3 are as follows:

• In figure 4.3, only hard task specification is considered due to thefull knowledge of the workspace;


• In figure 4.3, the FTS only updates when the next plan synthesisis executed. In the next plan synthesis, the robot localizes itselfand sets the current pose as the initial pose;

• Since the workspace is fully known, the initial plan has no needto be updated. The robot keeps reading the next move from theinitial plan until the task is changed.

4.1.3 Rqt

Rqt is a software framework in ROS that implements various GUI toolsin the form of plug-ins. All existing GUI tools can perform as dockablewindows within rqt. GUI tools can still run standalone, but rqt makesit easier to manage all various windows on the screen at the same time[11].

Figure 4.4: Rqt GUI [11].

Rqt provides a straight-forward way to monitor the real-time sys-tem. All running nodes, topics, messages and services are integratedin the same window and can be monitored simultaneously. Param-eters can be changed immediately and the simulation result can be


recorded as files just by clicking buttons. It is convenient for users todebug their codes.

As mentioned in the previous section, P-MAS-TG needs a robotmotion model(defined as a FTS) to calculate a valid task plan. In thetraditional way, the FTS needs to be predefined in the code, includingstates, actions, transition relations, atomic propositions and labellingfunctions. For every different workspace, there is a different set of pa-rameters that needs to be set up. In general, these parameters are or-ganized in specific data structure and it is a huge workload to modifythem.

With the aid of rqt, we can visualize the process of defining theFTS. The workspace can be loaded as a map file and all factors neededby FTS can be defined on the map directly by mouse click. P-MAS-TGcan work at the back-ends and read all parameters through the graphicuser interface. A more detailed description of the LTL planner plug-inimplementation can be seen in the next section.

4.2 LTL planner plug-in in rqt

The LTL planner plug-in in rqt follows the work flow shown in figure4.3. It resolves the problem caused by complex data structure of inputfiles and provides a GUI to design and monitor a robot plan. Threemain functions of the plug-in are map definition, plan synthesis and nav-igation. The plug-in can be approached on [37].

4.2.1 Map definition

Figure 4.5 shows the main user interface of map definition. In P-MAS-TG, the definition of a workspace has to be saved manually in thesource code in advance. Once the workspace has been changed, thecode has to be rewritten. It is inefficient and brings extra burdens fordebugging. With the use of the GUI, one can easily define a workspace(from the mapping algorithm by robots) and modify the definition. Inaddition, nodes on the map can be linked to poses of the robot, andedges of nodes can be linked to transition relations of poses.

Section 1: Map ViewerAfter the mapping step, the robot should have a map of the currentworkspace. The map viewer can show the map in the window with the


right resolution and one can define regions of interest directly on themap viewer by mouse click and drag. When the left button is clicked,an interest point will be recorded with its coordinate. Keep holdingthe left button and dragging it in one direction, the desired yaw angleof the robot at this point will also be recorded. After we release the leftbutton, the mentioned information will be saved and shown in section2, nodes viewer. When we click an existed node with right button, thenode will be deleted. The upper left of map viewer shows the currentposition and yaw angle of the mouse in real time. The upper rightshows two options for display abnormality.

Figure 4.5: The interface of map definition in LTL planner plug-in.

Section 2: Nodes ViewerThe defined nodes, which represent regions of interest, are shown innodes viewer. Every node contains details like name, position, andyaw angle. A new node can be added manually with input in the dia-logue box, but can only be deleted on map viewer by right click of themouse. After we have defined nodes on the map, edges among themwill be generated automatically and shown in section 3, edges viewer.Up to now the plug-in only generates two nearest edges of one node.

Section 3: Edges ViewerThe plug-in generates some edges among nodes automatically and shows


them in edges viewer. One can adjust it by manual operation like Addor Remove. Edges have no direction, which means r1<—->r2 has theexactly same meaning with r2<—->r1. After you have the satisfieddefinition, click Save button. Then the map information will be savedin a file and used by other plug-in elements. Reset and load buttonare also provided for convenience. Finally click Quit button to returnto the main window of the LTL planner plug-in.

The map definition function aims at solving the problem occurredat label 2 in figure 4.3. After the process, AP,L,→, Act, S of the FTScan be defined in a simple way.

Figure 4.6: The interface of plan synthesis in LTL planner plug-in

4.2.2 Plan synthesis

Figure 4.6 shows the user interface of plan synthesis. Immediatelyafter the LTL planner plug-in runs, this interface is shown by default.In P-MAS-TG, all related nodes on ROS have to be executed with thecommand line and are difficult to organize. With the help of the userinterface, nodes can be easily executed or terminated only by mouseclick. All of nodes are running in the back-ends. Users can focus on


observing the experiment result and analysis instead of struggling onthe organization of nodes.Section 1: Map SelectionThe map selection reads a map from a specific folder on the hard disk.Maps can be selected in the combo-box. After the selection, the mapis shown in section 2, map viewer. If the map contains predefined in-formation, nodes and edges are also shown. When clicking the DefineMap button, the plug-in jumps to the map definition module shownin 4.2.1.

Section 2: Map ViewerSimilar to section 1 in 4.2.1, a map viewer is provided on the main in-terface for displaying details of the map. But in this section the mapdefinition can not be modified. All operations directly on the mapneed to be done in the map definition module.

Section 3: Plan SynthesisAfter the map definition, we still need to gain two other necessary re-quirements in section 3 for plan synthesis. Users can specify the initialpose and the LTL task of the robot in the dialogue box. When all re-quirements are satisfied, press the button Plan Synthesis. Then theLTL planner in the package P-MAS-PG runs and gives a plan if thereexists. In figure 4.6 we can see that the initial pose is r2 and the task isvisiting r1, r2, r3 in order. Then the possible path is r2→ r1→ r2→ r3

and finally stay in r3. In the message box we can see that the plannergives the right plan.

Figure 4.7 shows the work flow of the Plan Execute button. Threenodes (task publisher, plan service and move action service) start afterthe button is pressed. The system is now waiting for the navigationrequest from users. Once users press the Navigation button in section4, the plan service sends the next goal to the move action service, andfinally the goal is sent to the move-base package in ROS to navigatethe robot.

Task Publisher reads the initial task from the dialogue box in section3 and publishes the task in ROS system.Plan Service subscribes task publisher for the LTL task and amcl nodefrom ROS for the initial pose of the robot. The plan service worksunder 3 types of request:


Figure 4.7: The work flow of plan execution.

• PlanSynthesis returns an available plan sequence with prefix-suffixstructure;

• FirstMove returns the first goal of the robot;

• NextMove returns the next goal of the robot.

Move Action Service receives the request from users for navigation.Once it has received the request, the move action service requests theplan sequence from the plan service and send the plan destinationby destination to the move-base package in ROS. Then the move-basepackage navigates the robot to the desired goal.

Section 4: Temporary Task and Navigation ControlAs mentioned in algorithm 3, the plug-in has the ability to deal with atemporary task inserted by humans. Based on algorithm 3 and figure4.7, the work flow of dealing with the temporary task can be seen infigure 4.8. If a temporary task is being involved(type the temporarytask in the Urgent Task box and click Execute button) while the robotis executing the initial task, the initial task will be terminated first.


Figure 4.8: The work flow of the temporary plan execution.

Then the task publisher restarts with the temporary task and the robotwill be navigated following the temporary task. When users terminatethe temporary task(click return button) or the temporary task has beenfinished, the task publisher restarts again with the initial task so thatthe robot returns to the previous task sequence.

The Navigation button works for starting the navigation processafter the right setup of elements in section 3. Navigation process canbe terminated any time when Stop button is clicked. Users can alsorun the code in Gazebo by clicking Simulation button instead of nav-igating on the real TurtleBot.

The plan synthesis module aims at solving the problem occurredat label 1 and 4 in figure 4.3. After the process, the initial state I of theFTS and the LTL formula φ can be defined in a simple way. Meanwhileit provides a simple hybrid controller of switching the task so that therobot can complete tasks globally(the initial task) and locally(the tem-porary task from humans) on line.


4.2.3 Navigation

The thesis is based on TurtleBot package in ROS and the simulationenvironment is on Gazebo. The main purpose of navigation is to drivethe robot from one region to another, and it is implemented based onlocalization package amcl and move_base package in ROS.

When plan synthesis has been completed, the plan will be pub-lished by task publisher mentioned in 4.2.2 when pressing the buttonPlan Execute. The plan is presented as a sequence of locations thatthe robot will drive to. The sequence will be divided into a combina-tion of single task and sent to the move_base package. After pressingthe button Navigation, the move_base package begins to receive thenext destination. Then the robot will be guided with a best path mov-ing from the current position to the desired destination. The currentposition is estimated by localization package amcl.

All the behaviour can be monitored in RViz, as shown in figure4.9. For simulation, Gazebo also provides a visualized interface fordisplaying the robot movement.

Figure 4.9: Navigation process in Gazebo and RViz

Chapter 5

Simulation and Result

In this thesis the simulation is conducted on Gazebo 2.x with Turtle-Bot package. The installation tutorial can be found on [44]. In thischapter, we firstly go through with a brief introduction of TurtleBotand Gazebo, then the simulation setup and the result are presented.Simulation 1 is for testing the basic function of the LTL plug-in withno temporary task involved, while simulation 2 contains a temporarytask when the robot is running to test the implementation of algorithm3.

Figure 5.1: TurtleBot 2

32

CHAPTER 5. SIMULATION AND RESULT 33

5.1 TurtleBot and Gazebo

TurtleBot is a low-cost, personal robot kit with open-source software.TurtleBot was created at Willow Garage by Melonee Wise and TullyFoote in November 2010 [29]. The TurtleBot kit consists of a mobilebase, 2D/3D distance sensor, laptop computer or SBC(Single BoardComputer), and the TurtleBot mounting hardware kit.

In this thesis, TurtleBot 2 is used and all functions are available andprovided in the form of ROS package.

Figure 5.2: gazebo 2.x

The concept of a high-fidelity simulator stemmed from the needto simulate robots in outdoor environments under various conditions,Gazebo offers the ability to accurately and efficiently simulate popu-lations of robots in complex indoor and outdoor environments [28].Gazebo has features such as dynamics simulation, realistic renderingof environments, robot models, sensor data and noise and even cus-tom plug-ins. Researchers can build up their desired world to simulatedifferent kinds of robots with respect to their research field.

Now the newest version is Gazebo 11.0. In this thesis Gazebo 2.xwithin ROS indigo is used. A newer Gazebo version has not beentested.

34 CHAPTER 5. SIMULATION AND RESULT

Figure 5.3: The simulation map

5.2 Simulation Setup

In this section, two simulations based on the same map are presented.The map has been defined by the LTL planner plug-in. The planneris from the P-MAS-TG and simulations are presented with TurtleBotpackage on Gazebo 2.x. For convenience, the task specification forboth simulations are given as the same, which is (23r2) ∧ (23r4),means surveillance between r2 and r4. In addition, the initial positionof the TurtleBot is given as the same as well, which is at r1.

Figure 5.4: Definition of the simulation map

All simulations are conducted on the map as shown in figure 5.3


and 5.4. The transition relations among different regions can be di-rectly read from the map.

Positions of every region point and the robot pose at the respectivepoint can be seen in figure 5.5.

As mentioned above, the LTL task specification is (23r2)∧ (23r4)and the initial point is at r1.

Figure 5.5: Regions of the simulation map

For the first simulation, run P-MAS-TG for plan synthesis and thenexecute the plan for navigation, observe the robot motion path in bothGazebo and RViz.

For the second simulation, a temporary task will be inserted whilethe initial task is being executed on the robot. Set up the temporarytask and click Execute and Return to see if the robot can switch itstask, observe the robot motion path in both Gazebo and RViz at thesame time.

5.3 Result

5.3.1 Result of Simulation 1

Figure 5.6: Plan synthesis of the simulation map


Figure 5.7: The expected robot motion in simulation 1

The plan synthesis given by P-MAS-TG is shown in figure 5.6. Withrespect to figure 5.5, the available plan is a sequence r1 → r2 → r1 →r4 → r1 → r2... and surveillance between r2 and r4. The expected ofrobot motion can be seen in figure 5.7.

Figure 5.8: The simulated robot motion in simulation 1

Notice that the initial task (23r2)∧ (23r4) is surveillance betweenr2 and r4. Therefore, in figure 5.6 we can see that the suffix of the planis repeating the path r1 → r2 → r1 → r4... so that the robot can visit r2and r4 over and over again.


The simulation result can be retrieved from [38]. The video per-forms a great lag due to the limited CPU resources, but as shown in5.8, it still shows that robot moves alongside the same trajectory asstated in figure 5.7.

5.3.2 Result of Simulation 2

Figure 5.9: Plan synthesis for the temporary plan.

In simulation 2, the initial task is exactly the same as that in sim-ulation 1, and the plan for the initial task can be seen in figure 5.6 aswell.

Figure 5.10: The expected robot motion in simulation 2

When the robot follows the plan r1 → r2 → r1 → r4 → r1 →


r2... as shown in 5.3.1 and runs between r1 and r4, a temporary task3(r3 ∧ 3r5), which means visiting r3 and r5 in order, is inserted. Thenew initial pose of the robot is the current pose received from amclpackage, and the new plan can be seen in figure 5.9.

As shown in the figure, the available plan is a sequence r4 → r1 →r3 → r1 → r5 and finally stay at r5. After the temporary task has beenfinished, the robot goes back to run the initial task again. The expectedof robot motion can be seen in figure 5.10.

Notice that in figure 5.9, the suffix of the plan is staying at the fi-nal destination r5. The reason is that the LTL task is visiting r3, r5 insequence, instead of surveillance.

Figure 5.11: The simulated robot motion in simulation 2

The simulation result can be retrieved from [39]. The video per-forms a great lag due to the limited CPU resources, but as shown in5.11, it still shows that robot moves alongside the same trajectory asstated in figure 5.10. When the robot has accomplished the temporarytask at r5, or user clicks Return, the robot goes back to follow the planof the initial task again.

5.3.3 Result Analysis

Both simulations show that the robot can successfully navigate itselffollowing the plan calculated by P-MAS-TG. The path the robot drives


from one point to the next point is decided by move_base package inROS. In addition, the robot needs to rotate itself for recovery everytime when executing a new plan. The reason is that in the simulationthe robot localizes itself with only amcl package in ROS, it may haveproblems of detecting the initial pose of the robot. The result can beimproved in the experiment because the real TurtleBot can localize it-self more accurately with the aid of more sensors.

Chapter 6

Experiment and Result

After the deployment of the LTL plug-in on both client computer andthe TurtleBot, two experiments are conducted to test the function ofthe plug-in. Experiment 1 is similar to simulation 2 for testing thecapability of the plug-in to deal with the temporary task inserted byhuman, while experiment 2 shows the ability of the robot to avoidhuman collision.

6.1 Experiment Setup

Figure 6.1: The projected map at the SML

The experiment is conducted at the Smart Mobility Lab(SML). Forconvenience of the comparison between the simulation and the exper-

40

CHAPTER 6. EXPERIMENT AND RESULT 41

iment, the projector in the SML is used to project the simulation map(figure 5.4) on the 5m x 5m testing ground, as shown in figure 6.1. Thesize of the projected map keeps the resolution of the simulation mapso that the robot motion behaviour on the projected map is similar tothat in the simulation. In addition, the map definition and the taskspecification are all the same as that in the simulation, which can beseen in 5.2.

6.1.1 Setup of Experiment 1

Experiment 1 is similar to simulation 2. The map definition is as shownin figure 5.4 and the regions of interest are as shown in figure 5.5.The task specification is also given as (23r2) ∧ (23r4), which meanssurveillance between r2 and r4. The initial position of the robot is alsoat r1.

In addition, a temporary task 3(r3 ∧3r5), which means visiting r3and r5 in order, is inserted when the initial task is running.

The purpose of experiment 1 is testing whether the plug-in can givea right task planning and switch the robot motion between the initialtask and the temporary task.

6.1.2 Setup of Experiment 2

In experiment 2, all settings keep the same as that in experiment 1 ex-cept for the task specification. Now the initial task is (23r2)∧(23r4)∧(23r5), which means surveillance among r2, r4 and r5, and no tempo-rary task is inserted. Another difference is that a human is walkingthrough the path of robot motion when task is being executed.

The purpose of experiment 2 is testing whether the robot has theability to avoid the human collision.

6.2 Result

6.2.1 Result of Experiment 1

The result of experiment 1 can be approached from [33] and [34]. Sinceall settings of experiment 1 are similar to that in simulation 2, the ex-pected robot motion trajectory can also be seen in figure 5.7.

42 CHAPTER 6. EXPERIMENT AND RESULT

In [34], we can see that the robot moves with the similar behaviourto that in the simulation, as shown in figure 6.2. The robot follows theplan r1 → r2 → r1 → r4 → r1 → r2... and finally surveillance betweenr2 and r4. When a temporary task 3(r3 ∧ 3r5) is inserted, the robotcancels the current goal and runs the temporary task in priority. Afterthe temporary task being finished or terminated, the robot goes backto the initial task again.

Figure 6.2: The real robot motion in experiment 1

Compared with the result of simulation 2 shown in figure 5.11, itis obviously that there is an extra path between r4 and r3 in figure 6.2.The reason is that when the temporary task is inserted, the robot iscloser to r4 than r1. Then the planner gives the plan r4 → r3 → r1 → r5instead to fulfil the lowest cost.

Notice that in [34], the robot does not move exactly in the regionof interest sometimes. It is because of the inaccuracy of the map andthe lower controller on the Turtlebot. Moreover, the robot is stuck androtates at the beginning when it moves from r2 to r1. The reason is thatthe robot needs to rotate for recovery of the localization.

6.2.2 Result of Experiment 2

The result of experiment 2 can be approached from [33] and [34]. Dueto the randomness of the human intervention, the robot motion path isunpredictable, but the expected robot behaviour can be that the plan-

CHAPTER 6. EXPERIMENT AND RESULT 43

ner will replan a new path to the next goal when a human blocking isdetected, as shown in 6.3.

Figure 6.3: The expected robot motion in experiment 2

As shown in 6.3, the red dash curve represents the original path.After the human is detected that blocking the path, the planner willreplan a new path as shown with blue curve to avoid the human colli-sion.

Figure 6.4: The real robot motion in experiment 2

Figure 6.4 shows clearly that the robot has the ability to avoid thehuman collision. When a human blocking is detected, the robot willcalculate a new available plan to reach the next goal.

44 CHAPTER 6. EXPERIMENT AND RESULT

Notice that in [34], when the robot moves from r1 to r4, the robotcan not find an available path. Therefore the robot cancels the currentgoal r4 and moves to the next goal r5

6.2.3 Result Analysis

Experiment 1 shows the similar result as that in simulation 2. P-MAS-TG can calculate a reasonable plan with the task specification and therobot can successfully navigate itself following the plan. In the videowe can see that sometimes the robot does not reach the region of inter-est exactly the same as defined. The reason is that the projected map isnot accurate enough and the robot would not localize itself precisely.Moreover, the lower notion controller of the Turtlebot also has the ac-curacy problem.

Experiment 2 shows that the robot has the ability to avoid the hu-man collision. When a human blocks the path, the robot stops andreplan a new path to finish the task specification.

Chapter 7

Conclusion and Future Work

7.1 Conclusion

In this thesis, the main features of the P-MAS-TG package are suc-cessfully visualized. The P-MAS-TG is an useful software packageto calculate the plan synthesis for given robot model following LTLspecifications. Due to the complexity of data structure used in the P-MAS-TG, it is not convenient to modify the FTS and LTL formula inthe source code. With LTL planner plug-in, one can organize the FTSdirectly on the map and specify LTL specification by directly typingin the box. In addition, plan synthesis and execution are integrated inthe plug-in as well. Researchers can complete simulation or naviga-tion of the robot within the plug-in, instead of multiple windows fordebugging.

Moreover, the LTL plug-in can be considered as a hybrid controllerto switch the task between the initial task and the temporary task in-serted by human. While the robot runs following the initial task, userscan insert a temporary task at any time and the robot will jump tothe temporary task. After users terminate the temporary task or it hasbeen finished, the robot can go back to the original task.

7.2 Future Work

In the future research, some improvements need to be implemented toreach the full features of the P-MAS-TG package.

45

46 CHAPTER 7. CONCLUSION AND FUTURE WORK

7.2.1 Full Features of P-MAS-TG

It is important to know that only part of the features involved in theP-MAS-TG are visualized in this thesis. In [13], infeasible task andpartially known workspace are handled but not involved in this thesis.

Moreover, not only moving task, but also other tasks like manipu-lation can also be modelled under the form of the finite-state transitionsystem and LTL formula.

Simulation 2 also shows some drawbacks of the LTL plug-in. Whenthe robot returns to the initial task, it can only execute the plan fromthe beginning, but not continue to the previous plan. In order to solvethe problem, we can save the visited location in the memory and callit back when the robot returns.

7.2.2 Multi-agent Collaboration

In the experiment part in [13], multi-agent robot system are shown andthe P-MAS-TG shows its ability to solve multi-agent problem. Theycan all be implemented in the future.

Bibliography

[1] Tomáš Babiak et al. “LTL to Büchi Automata Translation: Fastand More Deterministic”. In: Tools and Algorithms for the Con-struction and Analysis of Systems. Ed. by Cormac Flanagan andBarbara König. Berlin, Heidelberg: Springer Berlin Heidelberg,2012, pp. 95–109. ISBN: 978-3-642-28756-5.

[2] Christel Baier and Joost-Pieter Katoen. Principles of Model Check-ing (Representation and Mind Series). The MIT Press, 2008. ISBN:026202649X, 9780262026499.

[3] A. Balluchi et al. “Hybrid Control of Networked Embedded Sys-tems”. In: European Journal of Control 11.4 (2005), pp. 478–508.ISSN: 0947-3580. DOI: https://doi.org/10.1016/S0947-3580(05)71047-5.

[4] C. Belta, V. Isler, and G. J. Pappas. “Discrete abstractions forrobot motion planning and control in polygonal environments”.In: IEEE Transactions on Robotics 21.5 (Oct. 2005), pp. 864–874.ISSN: 1552-3098. DOI: 10.1109/TRO.2005.851359.

[5] C. Belta et al. “Symbolic planning and control of robot motion[Grand Challenges of Robotics]”. In: IEEE Robotics AutomationMagazine 14.1 (Mar. 2007), pp. 61–70. ISSN: 1070-9932. DOI: 10.1109/MRA.2007.339624.

[6] A Bhatia et al. “Motion Planning with Complex Goals”. In: 18(Oct. 2011), pp. 55–64.

[7] C.G. Cassandras and Stephane Lafortune. Introduction to DiscreteEvent Systems. Jan. 2010, p. 800. ISBN: 1441941193.

[8] Y. Chen et al. “Formal Approach to the Deployment of DistributedRobotic Teams”. In: IEEE Transactions on Robotics 28.1 (Feb. 2012),pp. 158–171. ISSN: 1552-3098. DOI: 10.1109/TRO.2011.2163434.

47

48 BIBLIOGRAPHY

[9] DariushForouher. Topics. http://wiki.ros.org/Topics.Jun 1, 2014.

[10] J. P. Desai, J. Ostrowski, and V. Kumar. “Controlling formationsof multiple mobile robots”. In: Proceedings. 1998 IEEE Interna-tional Conference on Robotics and Automation (Cat. No.98CH36146).Vol. 4. May 1998, 2864–2869 vol.4. DOI: 10.1109/ROBOT.1998.680621.

[11] DirkThomas. rqt. http://wiki.ros.org/rqt. Aug 30, 2016.

[12] DorianKurzaj. Messages. http://wiki.ros.org/Messages.Aug 26, 2016.

[13] Meng Guo. “Hybrid Control of Multi-robot Systems under Com-plex Temporal Tasks”. QC 20151204. PhD thesis. KTH, Auto-matic Control, 2015, pp. xiii, 164. ISBN: 978-91-7595-720-3.

[14] Meng Guo and Dimos V. Dimarogonas. “Multi-agent plan recon-figuration under local LTL specifications”. In: The InternationalJournal of Robotics Research 34.2 (2015), pp. 218–235. URL: https://doi.org/10.1177/0278364914546174.

[15] Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring Net-work Structure, Dynamics, and Function Using NetworkX. Jan. 2008.

[16] L. E. Kavraki et al. “Probabilistic roadmaps for path planningin high-dimensional configuration spaces”. In: IEEE Transactionson Robotics and Automation 12.4 (Aug. 1996), pp. 566–580. ISSN:1042-296X. DOI: 10.1109/70.508439.

[17] KenConley. Nodes. http://wiki.ros.org/Nodes. Feb 3,2012.

[18] KenConley. Services. http://wiki.ros.org/Services. Feb3, 2012.

[19] M. Kloetzer and C. Belta. “Automatic Deployment of DistributedTeams of Robots From Temporal Logic Motion Specifications”.In: IEEE Transactions on Robotics 26.1 (Feb. 2010), pp. 48–61. ISSN:1552-3098.

[20] J.C. Latombe. Robot Motion Planning. The Springer InternationalSeries in Engineering and Computer Science. Springer US, 2012.ISBN: 9781461540229. URL: https://books.google.se/books?id=nQ7aBwAAQBAJ.

BIBLIOGRAPHY 49

[21] Steven M. LaValle and Jr. James J. Kuffner. “Randomized Kin-odynamic Planning”. In: The International Journal of Robotics Re-search 20.5 (2001), pp. 378–400. DOI: 10.1177/02783640122067453.eprint: https://doi.org/10.1177/02783640122067453.URL: https://doi.org/10.1177/02783640122067453.

[22] S. R. Lindemann, I. I. Hussein, and S. M. LaValle. “Real TimeFeedback Control for Nonholonomic Mobile Robots With Ob-stacles”. In: Proceedings of the 45th IEEE Conference on Decision andControl. Dec. 2006, pp. 2406–2411. DOI: 10.1109/CDC.2006.377631.

[23] J. Lygeros et al. “Dynamical properties of hybrid automata”. In:IEEE Transactions on Automatic Control 48.1 (Jan. 2003), pp. 2–17.ISSN: 0018-9286. DOI: 10.1109/TAC.2002.806650.

[24] Meng Guo. Planner for Multiple Agent System with Temporal Goals.https://github.com/MengGuo/P_MAS_TG. May 10, 2018.

[25] S. Misra and B. J. Oommen. “Dynamic algorithms for the short-est path routing problem: learning automata-based solutions”.In: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cy-bernetics) 35.6 (Dec. 2005), pp. 1179–1192. ISSN: 1083-4419. DOI:10.1109/TSMCB.2005.850180.

[26] Adam Morecki and Józef Knapczyk. “Robot Task Planning”. In:Basics of Robotics: Theory and Components of Manipulators and Robots.Ed. by Adam Morecki and Józef Knapczyk. Vienna: SpringerVienna, 1999, pp. 319–377. ISBN: 978-3-7091-2532-8. DOI: 10 .1007/978- 3- 7091- 2532- 8_12. URL: https://doi.org/10.1007/978-3-7091-2532-8_12.

[27] Muhayyuddin, Aliakbar Akbari, and Jan Rosell. “Physics-basedMotion Planning with Temporal Logic Specifications”. In: IFAC-PapersOnLine 50.1 (2017). 20th IFAC World Congress, pp. 8993–8999. ISSN: 2405-8963. DOI: https://doi.org/10.1016/j.ifacol.2017.08.1578.

[28] Open Source Robotics Foundation. Gazebo. http://gazebosim.org/. 2014.

[29] Open Source Robotics Foundation, Inc. Turtlebot. https : / /www.turtlebot.com/. May 21, 2016.

50 BIBLIOGRAPHY

[30] Patrick Caughill. Google’s Self-Driving Division Is Expanding ItsFleet of Autonomous Vehicle. https://futurism.com/waymo-fiat-autonomous-vehicles/. January 31, 2018.

[31] A. Puri and P. Varaiya. “Decidable hybrid systems”. In: Math-ematical and Computer Modelling 23.11 (1996), pp. 191–202. ISSN:0895-7177. DOI: https://doi.org/10.1016/0895-7177(96)00072-6.

[32] ROS contributors. About ROS. http://www.ros.org/about-ros/. May 10, 2018.

[33] Tengfan Lin. Experiment 1 on the Rviz. https://youtu.be/vevWsvdCQkM. Aug 5, 2018.

[34] Tengfan Lin. Experiment 1 on the Turtlebot. https://youtu.be/31x910kzphY. Aug 5, 2018.

[35] Tengfan Lin. Experiment 2 on the Rviz. https://youtu.be/t_gU2uz87Ns. Aug 5, 2018.

[36] Tengfan Lin. Experiment 2 on the Turtlebot. https://youtu.be/mkL4OFajsyQ. Aug 5, 2018.

[37] Tengfan Lin. LTL-planner-plugin-rqt. https://github.com/Dayelyn/LTL-planner-plugin-rqt. May 22, 2018.

[38] Tengfan Lin. Simulation 1. https://youtu.be/5x-abs5TcOk.Aug 5, 2018.

[39] Tengfan Lin. Simulation 2. https://youtu.be/AOBFVG8aZlA.Aug 5, 2018.

[40] Moshe Vardi and Pierre Wolper. “An Automata-Theoretic Ap-proach to Automatic Program Verification (Preliminary Report)”.In: (Jan. 1986), pp. 332–344.

[41] C. I. Vasile and C. Belta. “Sampling-based temporal logic pathplanning”. In: 2013 IEEE/RSJ International Conference on Intelli-gent Robots and Systems. Nov. 2013, pp. 4817–4822. DOI: 10.1109/IROS.2013.6697051.

[42] Wikipedia contributors. Amazon Prime Air. https://en.wikipedia.org/wiki/Amazon_Prime_Air. April 2, 2018.

[43] Wikipedia contributors. Autonomous car. https://en.wikipedia.org/wiki/Autonomous_car. May 5, 2018.

BIBLIOGRAPHY 51

[44] Yoonseok Pyo. TurtleBot. http://wiki.ros.org/Robots/TurtleBot. Apr 8, 2018.

Appendix A

User Manual

A.1 Deployment

A.1.1 Prerequisites

• Ubuntu 14.04 with ROS indigo Desktop-Full installed;

• The package is based on the Turtlebot, full Turtlebot packageshould be installed (including simulation environment);

• P-MAG-TS is used. Please follow the instruction in [24];

• If you have failed in running test code in P-MAG-TS, please use1.x version of networkx.

A.1.2 Installation

Download or pull the code from [37] to your ROS workspace, then

$ cd ~/your_ROS_workspace$ catkin_make

A.1.3 Execution

Before running the rqt, you have to ensure roscore is running. If not,open a terminal and type

$ roscore

then open a new terminal and type

52

APPENDIX A. USER MANUAL 53

$ r q t

Then from Plugins choose LTL and click LTL debugger to executethe plug-in.

A.2 Map Definition

There are five maps provided by default. Users can select a map byclicking Select a Map. If users would like to use their own map, pleasecopy map image and .yaml file into your_ros_workspace/src/rqt_ltl/map/testmap.

Sometimes users will have map.pgm after mapping step. The bestway to use the plug-in to to exchange .pgm to .png, otherwise themap viewer may have problem on displaying the map.

After the map selection, click Define Map to open the window ofmap definition. Then follow the instruction in 4.2.1, the map definitioncan be finished. Do not forget saving the definition before quit.

A.3 Plan Synthesis

Users need to specify the initial pose and the linear temporal task forthe robot to move to plan synthesis. By clicking Default the plug-inwill automatically specify them by default setting.

If users would like to specify the initial pose and the LTL task bythemselves, please input them into the dialogue box and click PlanViewer to choose Custom.

Then click Plan Synthesis, the plan calculated by the P-MAS-TGwill be shown in the right part of the window.

The next step is clicking Plan Execution, then the necessary infor-mation will be saved and the task is being publishing in the ROS sys-tem.

A.4 Simulation

For the simulation, simply just by clicking Simulation. The start set-tings of simulation can be modified in /your_ros_workspace/src/tb_simulation/launch/tbgazebo.launch. If there are errors

54 APPENDIX A. USER MANUAL

when gazebo is starting, please follow the error information and mod-ify related options.

After the start of Gazebo, click Navigation to move the robot fol-lowing the LTL task. Click Stop to terminate the navigation.

If users would like to insert a temporary task, please input the LTLtask in Urgent Task and click Execute. The robot will stop from thecurrent task and jump to the temporary task. If users would like to goback to the initial task, please click Return or waiting for the end ofthe temporary task.

A.5 Experiment

For the experiment, please connect to the Turtlebot right after clickingPlan Execute with following steps:

• Edit the .bashrc for both user’s computer and the Turtlebot toexport ROS_ MASTER_ URI with the IP address of the user’scomputer

• Run Turtlebot bringup on the Turtlebot:

$ roslaunch t u r t l e b o t _ b r i n g u p minimal . launch −−screen

• Save the map on the Turtlebot and export TURTLEBOT_ MAP_FILE to your directory:

$ export TURTLEBOT_MAP_FILE=/your_direc tory/map . yaml

• Run amcl on the Turtlebot:

$ roslaunch t u r t l e b o t _ n a v i g a t i o n amcl_demo . launch

• Run Rviz on the user’s computer to observe the robot navigation:

$ roslaunch t u r t l e b o t _ r v i z _ l a u n c h e r s view_navigation . launch −−screen

• Click Navigation to move the robot. Other operations are thesame as that in the simulation.

TRITA EECS-EX-2018:538

ISSN 1653-5146

www.kth.se

a gui design of robot motion and task planning based on linear …1249798/... · 2018. 9. 20. ·...

Documents