towards model-predictive control for aerial pick-and · pdf filetowards model-predictive...

6
Towards Model-predictive Control for Aerial Pick-and-Place Gowtham Garimella 1 and Marin Kobilarov 1 Abstract— This paper considers pick-and-place tasks using aerial vehicles equipped with manipulators. The main focus is on the development and experimental validation of a nonlinear model-predictive control methodology to exploit the multi-body system dynamics and achieve optimized performance. At the core of the approach lies a sequential Newton method for unconstrained optimal control and a high-frequency low-level controller tracking the generated optimal reference trajectories. A low cost quadrotor prototype with a simple manipulator extending more than twice the radius of the vehicle is designed and integrated with an on-board vision system for object tracking. Experimental results show the effectiveness of model- predictive control to motivate the future use of real-time optimal control in place of standard ad-hoc gain scheduling techniques. I. I NTRODUCTION This work develops a nonlinear model-predictive control (NMPC) approach to enable agile pick-and-place capabili- ties for aerial vehicles equipped with manipulators. Aerial manipulation using vertical take-off and landing (VTOL) vehicles is a relatively new research area with a potential for various novel applications such as coordinated assembly, construction, and repair of structures at high altitudes, or op- erating in difficult-to-access, remote, or hazardous locations to e.g. install sensors or obtain samples. Autonomous control of such system is challenging primarily due to disturbances from interactions with the environment, due to additional dynamics caused by a moving manipulator, and due to difficulties associated with dexterous manipulation. Initial work related to aerial manipulation included slung load transportation with helicopters [6], [20], grasping with novel adaptive end-effectors [27], [26], construction using teams of quadcopters [18], or pole balancing tasks [7]. More recently, there has been a focus on autonomous construction and environment interaction, with initial demonstrations in laboratory settings. The Aerial Robotics Cooperative Assem- bly System (ARCAS) project [2], [12], [8], Mobile Manip- ulating Unmanned Aerial Vehicle project [25], [16], [17] and Airobots project [1], [31] have demonstrated complex manipulation and assembly tasks using multiple degrees of freedom manipulators. Other important developed capabil- ities include telemanipulation [22], [11] or avian-inspired agile grasping [30]. In addition to control-related challenges, accurate pose estimation of objects is of central impor- tance and has been considered through image-based visual servoing [29] and marker-based pose computation [2], [7]. Real-time recognition and aerial manipulation of arbitrary 1 Gowtham Garimella and Marin Kobilarov are with the Department of Mechanical Engineering, Johns Hopkins University, 3400 N Charles Str, Baltimore, MD 21218, USA ggarime1|[email protected] Fig. 1. a) a prototype quadrotor with manipulator, b) schematic model, c) a computed optimal trajectory viewed in Robot Operating System (ROS). unengineered objects in natural settings remains largely an open problem. Control strategies for aerial manipulation can be divided into coupled which consider the full multi-body system model [19], [14], [24], and decoupled based on separate controllers for the base body and manipulator [28]. The key difference is that the decoupled approach treats external forces from the arm or environment as disturbances to be compensated by the vehicle. In this paper, we propose an optimal control algorithm for generating reference trajectories to pick an object using aerial vehicle. Experimental verification has been performed using a minimalist low-cost system based on a two-degree of freedom manipulator with a simple gripper. The task is made challenging by using a monocular camera to recognize and track the target object. To facilitate recognition, objects

Upload: vanmien

Post on 11-Mar-2018

223 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

Towards Model-predictive Control for Aerial Pick-and-Place

Gowtham Garimella1 and Marin Kobilarov1

Abstract— This paper considers pick-and-place tasks usingaerial vehicles equipped with manipulators. The main focus ison the development and experimental validation of a nonlinearmodel-predictive control methodology to exploit the multi-bodysystem dynamics and achieve optimized performance. At thecore of the approach lies a sequential Newton method forunconstrained optimal control and a high-frequency low-levelcontroller tracking the generated optimal reference trajectories.A low cost quadrotor prototype with a simple manipulatorextending more than twice the radius of the vehicle is designedand integrated with an on-board vision system for objecttracking. Experimental results show the effectiveness of model-predictive control to motivate the future use of real-time optimalcontrol in place of standard ad-hoc gain scheduling techniques.

I. INTRODUCTION

This work develops a nonlinear model-predictive control(NMPC) approach to enable agile pick-and-place capabili-ties for aerial vehicles equipped with manipulators. Aerialmanipulation using vertical take-off and landing (VTOL)vehicles is a relatively new research area with a potentialfor various novel applications such as coordinated assembly,construction, and repair of structures at high altitudes, or op-erating in difficult-to-access, remote, or hazardous locationsto e.g. install sensors or obtain samples. Autonomous controlof such system is challenging primarily due to disturbancesfrom interactions with the environment, due to additionaldynamics caused by a moving manipulator, and due todifficulties associated with dexterous manipulation.

Initial work related to aerial manipulation included slungload transportation with helicopters [6], [20], grasping withnovel adaptive end-effectors [27], [26], construction usingteams of quadcopters [18], or pole balancing tasks [7]. Morerecently, there has been a focus on autonomous constructionand environment interaction, with initial demonstrations inlaboratory settings. The Aerial Robotics Cooperative Assem-bly System (ARCAS) project [2], [12], [8], Mobile Manip-ulating Unmanned Aerial Vehicle project [25], [16], [17]and Airobots project [1], [31] have demonstrated complexmanipulation and assembly tasks using multiple degrees offreedom manipulators. Other important developed capabil-ities include telemanipulation [22], [11] or avian-inspiredagile grasping [30]. In addition to control-related challenges,accurate pose estimation of objects is of central impor-tance and has been considered through image-based visualservoing [29] and marker-based pose computation [2], [7].Real-time recognition and aerial manipulation of arbitrary

1Gowtham Garimella and Marin Kobilarov are with the Department ofMechanical Engineering, Johns Hopkins University, 3400 N Charles Str,Baltimore, MD 21218, USA ggarime1|[email protected]

Fig. 1. a) a prototype quadrotor with manipulator, b) schematic model, c)a computed optimal trajectory viewed in Robot Operating System (ROS).

unengineered objects in natural settings remains largely anopen problem.

Control strategies for aerial manipulation can be dividedinto coupled which consider the full multi-body systemmodel [19], [14], [24], and decoupled based on separatecontrollers for the base body and manipulator [28]. Thekey difference is that the decoupled approach treats externalforces from the arm or environment as disturbances to becompensated by the vehicle.

In this paper, we propose an optimal control algorithmfor generating reference trajectories to pick an object usingaerial vehicle. Experimental verification has been performedusing a minimalist low-cost system based on a two-degreeof freedom manipulator with a simple gripper. The task ismade challenging by using a monocular camera to recognizeand track the target object. To facilitate recognition, objects

Page 2: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

are engineered with LED markers correspoding to knownfeatures. A detailed nonlinear model is employed by theoptimal control framework to capture the interaction betweenthe arm and quadcopter. Currently due to computationallimitations, NMPC operates at 10Hz and is not used for real-time control. Hence a high frequency nonlinear controlleris coupled with the optimal control framework to track thereference trajectories.

The paper is organized as follows. The dynamical multi-body system modeling and numerical optimal control ap-proach are described in §II and §III, respectively. Then weproceed to describe the experiments conducted to validatethe optimal controller in §IV. Finally we provide the resultsof the experiments conducted and discuss future work in §V.

II. SYSTEM MODELING

The aerial robot is modeled as a free-flying multi-bodysystem consisting of n + 1 interconnected rigid bodiesarranged in a tree structure. The configuration of body #i isdenoted by gi ∈ SE(3) and defined as

gi =

(Ri pi0 1

), g−1i =

(RT

i −RTi pi

0 1

).

where pi ∈ R3 denotes the position of its center of massand and Ri ∈ SO(3) denotes its orientation. Its body-fixedangular and linear velocities are denoted by ωi ∈ R3 andνi ∈ R3. The pose inertia tensor of each body is denoted bythe diagonal matrix Ii defined by

Ii =

(Ji 00 miI3,

)where Ji is the rotational inertia tensor, mi is its mass,and In denotes the n-x-n identity matrix. The system has njoints described by parameters r ∈ Rn. Following standardnotation [23], the relative transformation between the basebody#0 and body#i is denoted by g0i : Rn → SE(3), i.e.

gi = g0g0i(r).

The control inputs u ∈ U ⊂ Rm=n+4 denote the four rotorspeeds squared and the n joint torques. More specifically,ui = Ω2

i for i = 1, . . . , 4 where Ωi is the rotor speed ofthe i-th rotor, and u4+i denotes the i-th joint torque, fori = 1, . . . , n.

The configuration of the system is thus given by q ,(g, r) ∈ Q , SE(3) × Rn, where g ∈ SE(3) is a chosenreference frame moving with the robot. In this work we takethe base body as a moving reference, i.e. g ≡ g0. The velocityof the system is given by v , (V, r) ∈ R6+n, where V ∈ R6

denotes the body-fixed velocity of the moving frame g andr ∈ Rn denotes the joint angle velocities. The base velocitysatisfies V = g−1g where the “hat” operator V for a givenV = (ω, ν) is defined by

V =

[ω ν

01×3 0

], ω =

0 −w3 w3

w3 0 −w1

−w2 w1 0

. (1)

With these definitions, the full state of the system is x ,(q, v) ∈ X , Q× R6+n.

Continuous Equations of Motion: The coordinates forour setting are q = (g, r) where the pose g ∈ SE(3) and rrepresents joint parameters. For optimal control purposes, itis necessary to avoid Euler angle singularities and, in addi-tion, it is advantageous to avoid unit quaternion constraints.To achieve this, the dynamics is defined directly on statespace X as:

g = gV (2)M(r)v + b(q, v) = Bu, (3)

where the mass matrix M(r), bias term b(q, v), and constantcontrol matrix B are computed analogously to standardmethods such as the articulated composite body algorithm [5]or using spatial operator theory [10]. With our coordinate-free approach the mass matrix in fact only depends on theshape variables r rather than on q and for tree-structuredsystems can be computed readily according to

M(r) =

I0 +n∑

i=1

ATi IiAi

∑ni=1A

Ti IiJi∑n

i=1 JTi IiAi

∑ni=1 J

Ti IiJi

(4)

using the adjoint notation Ai := Adg−10i (r), and Jacobian

Ji :=∑n

j=1[g−10i (r)∂rjg0i(r)]∨,where g0i(r) is the relative

transformation from the base body to body #i and Ii is theinertia tensor of body #i [23].

The bias term b(q, v) encodes all Coriolis, centripetal,gravity, and external forces. Finally, for a quadrotor modelthe constant control matrix B has the form

B =

0 −lkt 0 lkt−lkt 0 lkt 0km −km km −km0 0 0 00 0 0 0kt kt kt kt

06×`

0`×6 I`×`

,

where l, kt, km are known constants. This can be easilyextended to other multi-rotor configurations.

Discrete Dynamics: For computational purposes weemploy discrete-time state trajectories x0:N , x0, . . . , xNat equally spaced times t0, . . . , tN ≡ tf with time step∆t =

tf−t0N . The discrete state at index k approximates the

continuous state at time tk = t0 + k∆t, i.e. xk ≈ x(tk) andis defined by xk = (gk, rk, Vk,∆rk), where ∆rk denotes thejoint velocities at k-th stage. A simple discrete-time versionof the continuous dynamics (2)–(3) is then employed:

gk+1 = gk cay (∆tVk+1) , (5)rk+1 = rk + ∆t∆rk+1, (6)

M(rk)vk+1 − vk

∆t+ b(qk, vk) = Buk. (7)

This is a first-order semi-implicit method since one firstupdates the velocity vk+1 using the dynamics (7) and thenupdates the configuration using the kinematics (5)–(6). Themethod requires small time-steps to ensure stability (∆t ≤

Page 3: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

100ms is sufficient for the aerial systems considered), higher-order methods are also possible [15], [13].

Note that the base pose update (5) is performed using theCayley map cay : R6 → SE(3) defined (see e.g. [15]) by

cay(V ) =

[I3+ 4

4+‖ω‖2

(ω+ ω2

2

)2

4+‖ω‖2 (2I3+ω) ν

0 1

], (8)

instead of the more standard exponential map on SE(3) [23],[4] since it is an accurate and efficient approximation, i.e.cay(V ) = exp(V ) + O(‖V ‖3), it preserves the groupstructure, and has particularly simple to compute derivatives.Its inverse is denoted by cay−1 : SE(3)→ R6 and is definedfor a given g = (R, p), with R 6= −I , by

cay−1(g) =

[[−2(I +R)−1(I −R)]∨

(I +R)−1p

].

III. MODEL PREDICTIVE CONTROL

To achieve agile pick-and-place motions we employmodel-predictive control to optimize future trajectories overthe interval [t0, tf ] where t0 is the current time and tf is aspecified moving horizon. A typical horizon tf − t0 for theconsidered aerial maneuvers is between 2 and 5 seconds. Twomethods for unconstrained optimal control are consideredin view of their capacity for near real-time performance:a simple Gauss-Newton shooting method and a StagewiseNewton sweep method.

A. Optimal Control Formulation

The NMPC problem can be generally formulated as theminimization of:

J(x0:N , u0:N−1) , LN (xN ) +

N−1∑k=0

Lk(xk, uk), (9)

subject to: xk+1 = fk(xk, ui), uk ∈ U (10)

where fk encodes the integrator (5)–(7) and U definesthe admissible control set. The stage-wise cost penalizesdeviation from a desired nominal state xd and controls udand is given by

Lk(xk, uk) =1

2‖xk − xd‖2Qk

+1

2‖uk − ud‖2Rk

, (11)

while the terminal cost is defined by

LN (xN ) =1

2‖xN − xf‖2Qf

, (12)

where Qk ≥ 0, Qf > 0, Rk > 0 are appropriately chosendiagonal matrices to tune the vehicle behavior while reachinga desired final state xf . In the aerial robot applicationthe matrix Qk contains non-zero terms corresponding to adesired velocity only.

B. Overloading ± operator on the group SE(3)

Numerical optimal control is based on vector calculuswhich is not directly applicable to states x = (g, r, v)containing matrix elements g ∈ SE(3). Hence, we use vectoroperators with analogous “retract” and “lift” operators onSE(3).

The lift operator on SE(3) is equivalent to operator minus(·)− (·) : SE(3)× SE(3)→ R6

gb − ga = V ⇐⇒ cay−1(g−1a gb) = V,

gb − ga , cay−1(g−1a gb) = V ∈ R6, (13)

or practically speaking the differences between two posesapproximately equals the constant body-fixed velocity Vwith which ga moves to align with gb after one unit of time.The retract operator on SE(3) is equivalent to plus or minus(·)± (·) : SE(3)× R6 → SE(3) according to

ga ± V , gacay(±V ) = gb ∈ SE(3), (14)

i.e. adding/subtracting a vector V to/from the matrix ga isinterpreted as shifting ga using a unit-time transformationwith constant body-fixed velocity V . These operations areliterally overloaded in our C++ implementation when oneattempts to subtract two SE(3) matrices, or add/subtract avector V to/from a matrix ga. With these definitions, theerrors xi− xd and xN − xf appearing in the costs (11),(12)are defined using the lift operator (13) so that, e.g. the latterwith xf = (gf , rf , vf ) should be understood as

xN − xf ≡

cay−1(g−1f gN )

rN − rfvN − vf

.C. Gauss-Newton shooting method

One of the simplest, often overlooked, but surprisingly ef-fective methods for solving the optimal control problem (9)–(10) is a shooting method exploiting the least-squares natureof the costs (11)–(12). It is formulated by parametrizing thediscrete control trajectory u0:N−1 using a vector ξ ∈ R`≤Nm,encoded through the functions uk = φk(ξ) for each k =0, . . . , N − 1. For instance, ξ could contain the knots ofa B-spline from which each uk is extracted. The simplestparametrization is to simply set ξ = u0:N−1. Using thedynamics each state can be expressed as a function of ξwhich is encoded through the functions xk = ψk(ξ) for k =0, . . . , N . The cost is then expressed as J(ξ) = 1

2h(ξ)Th(ξ),where h : R` → RN(m+n+6) is given by

h(ξ) =

√R0 (φ0(ξ)− ud)√Q1 (ψ1(ξ)− xd)√R1 (φ1(ξ)− ud)

...√QN−1 (ψN−1(ξ)− xd)√RN−1 (φN−1(ξ)− ud)√Qf (ψN (ξ)− xf )

.

Since Ri > 0 the Jacobian ∂h(ξ) is guaranteed to be full rankand one can apply a Gauss-Newton iterative method directlyto update ξ → ξ+δξ where δξ = −(∂gT∂g)−1∂gT g. In ad-dition, the Jacobian has a lower-triangular structure that canbe exploited in the Cholesky GN solution. The complexity ofthis method is still O(`3) which is only acceptable for small`, e.g. ` ≤ 100 in order to achieve real-time performance.The key advantage of the GN approach is its simplicity and

Page 4: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

robustness by employing standard regularization and line-search techniques [3].

A more efficient method with complexity O(N(n + m))that exploits the recursive optimal control problem structureis presented next.

D. Stagewise Newton and Differential Dynamic Program-ming

The second optimal control method used in this work isbased on a coordinate-free recursive NMPC formulation [13],[15] for optimization on state spaces with Lie group structuresuch as SE(3). The particular method we employ is StagewiseNewton (SN) [3] which is also closely related to Differentialdynamic programming (DDP) [9].

Stagewise methods explicitly require the linearization ofthe cost and of the dynamics. On non-Euclidean manifoldsX such linearization is achieved using trivialized variationsand gradients [13]. In particular, for the class of systemsconsidered in this work, the linearized discrete dynamicstakes the form

dxk+1 = Akdxk +Bkδuk, (15)

with dxk , (dgk, δrk, δvk) where dg , (g−1δg)∨ ≡((RT δR)∨, RT δp) ∈ R6 is the trivialized variation onSE(3). Similarly, the trivialized gradient dgL ∈ R6 of afunction L : SE(3)→ R is defined by

dgL , ∇V

∣∣∣V=0

L(g cay(V )), (16)

for some V ∈ R6. With these definitions, any standarditerative optimization method such as SQP, SN, or DDP canbe applied by replacing the standard gradients ∇gL, ∇2

gLand variations δg, with the trivialized gradients dgL, d2gLand trivialized variations dg.

Finite-difference linearization of the dynamics.: Sincethe resulting multi-body dynamics (5)–(7) has a complexnonlinear form, we employ finite differences for computingthe Jacobians Ak and Bk. The default choice is centraldifferences:

Aik ≈

f(xk + εei, uk)− f(xk − εei, uk)

2ε,

Bjk ≈

f(xk, uk + εej)− f(xk, uk − εej)2ε

,

for i = 1, . . . , n + 6, and j = 1, . . . ,m, where eachei is a standard basis unit vector with only one non-zeroelement at its i-th component. We again emphasize that the+ and − signs above should be interpreted as the overloadedoperators (13),(14) whenever elements of SE(3) are involved.

Closed-form cost gradients.: The trivialized gradientand Hessian of Li are straightforward to compute and onlyrequire an extra term to account for the Cayley map. Theyare given by:

dLk =

[dcay−1(−∆k) 0

0 I

]TQk(xk − xd), (17)

d2Lk≈[

dcay−1(−∆k) 00 I

]TQk

[dcay−1(−∆k) 0

0 I

], (18)

where ∆k = cay−1(g−1d gk) for each k = 0, . . . , N − 1.Equivalent expressions also hold for the gradients of LN ,with gd replaced by gf , and Qk with Qf . Note that forsimplicity the Hessian was approximated by ignoring thesecond derivative of cay. The trivialized Cayley derivativedenoted by dcay(V ) for some V = (ω, ν) ∈ R6 is defined(see e.g. [15]) as

dcay(V )=

[2

4+‖ω‖2 (2I3 + ω) 031

4+‖ω‖2 ν(2I3 + ω) I3+ 14+‖ω‖2 (2ω+ω2)

],

(19)

it is invertible and its inverse has the simple form

dcay−1(V ) =

[I3 − 1

2 ω + 14ωω

T 03− 1

2

(I3 − 1

2 ω)ν I3 − 1

2 ω

]. (20)

The linearized dynamics (15), cost gradients (17) and Hes-sians (18) can now be used as the ingredients of a standardStagewise Newton algorithm [3] as detailed in [13].

IV. EXPERIMENTAL SETUP

In this section the hardware and software architecturerequired for running the manipulation experiments is out-lined. Later, the experiments conducted and the NMPC basedreference trajectory approach are described.

Fig. 2. a) Experimental arena showing the object to grab (black bottle)and led markers b) Led markers as seen from onboard camera

A. Hardware

Our prototype platform is based on the 3DRobotics quad-copter capable of lifting a payload of 1Kg, the Pixhawkautopilot board [21] for low-level attitude and thrust control,and the Odroid XU+E bare board computer for running var-ious control algorithms. The NaturalPoint OptiTrack MotionCapture System has been used for estimating the attitude andposition of the quadcopter in the world frame. A lightweightcamera (PointGrey Firefly model) is installed onboard forproviding the relative position of the target object in thereference frame of the quadcopter. A custom manufacturedlightweight arm along with a 3D printed gripper has beeninstalled on the quadcopter to grasp the object.

B. Experimental Scenario

The experimental scenario in Figure 2 shows the object ofinterest and LED markers which are used by the onboardcamera to detect and track the object. The experimentrequires the quadcopter to detect the marker, fly to a specifiedlocation in front of it, and retrieve the object placed on astand. This is a challenging task, since the quadcopter has

Page 5: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

to extend the arm farther beyond it’s enverlope to grab thetarget object.

Fig. 3. Optimal control trajectory followed by the quadcopter. The solidline represents the desired trajectory and dashed line represents the actualtrajectory followed

C. NMPC-based reference trajectory generation

In this approach, we compute a reference trajectory for thecombined system of the quadcopter and arm using StagewiseNewton method described in § III-D. The optimal controlleris used in open-loop to compute a reference trajectory forthe quadcopter and the arm. The object position is frozenonce the quadcopter starts executing the reference trajectory.The computed trajectory contains the full state (position,orientation and body fixed velocities) of the quadcopter,full state of the arm (joint positions and velocities) andthe controls needed to achieve them. The desired positionand velocity of the quadcopter is fed into the feedbacklinearization based controller and the desired joint angles andvelocities for the arm are achieved through the PID controlleron the servo motors. The quadcopter is able to track thetrajectory closely as shown in Figure 3. The optimal controlapproach allows for faster actuation of the arm without losingaccuracy. Since both the arm and the quadcopter execute theirrespective trajectories simultaneously, the object is retrievedin a shorter time interval.

V. RESULTS AND DISCUSSION

The series of pictures in Figure 4 show the quadcopterflying to the marker and retrieving the object. The upperhalf represents the approach to grasp the object and the lowerhalf shows the retrieval of the object (See attached video formore detail). The average time for retrieval using manualreference trajectories (Fig[5]) is around 15 seconds. UsingNMPC-based trajectory tracking reduces the time taken tograsp the object to 5 seconds. This confirms that usingoptimal reference trajectories is superior to manual referencetrajectories.

The tip positions for various starting positions convergingto the target location (red cuboid) have been plotted inFigure 5. Since the non-optimal manual reference trajectorydoes not account for the dynamics of the arm, the tippositions are not smooth and take longer time to convergeto the grasping location. On the other hand, the tip positionfor the case using optimal control (denoted by black dashed

line) is smoother and still converges with the same accuracy.Following the optimal control based reference trajectory hasenabled us to actuate both the quadcopter and the armsimultaneously to grasp the object quickly.

Fig. 5. Comparison of end-effector tip position trajectories for PID andMPC starting from from different initial positions marked with black dots.

There are many challenges faced during the manipula-tion tasks described above. The quadcopter position canbe estimated based on the pose of the markers from theonboard camera. But this estimate turned out to be noisyand is dependent on the distance between the camera andmarkers. Thus a motion capture system is used to provide areliable estimate of the quadcopter state. Since we are usinga feedback linearization based controller for the quadcopter,we are not using optimal control to its full capacity ofdirectly commanding the quadcopter motors. This explainsthe slight discrepancies between the actual and the desiredquadcopter trajectories shown in Figure 3.

Future work will focus on improving the optimal controlalgorithm to achieve a real-time performance. By runninga fast optimal control algorithm, we expect to bypass thefeedback linearizing control and directly use the optimalcontroller to send the low-level motor commands. To enablemarkerless object grasping in an outdoor environment, wewill rely on robust feature based object recognition andtracking. Combining real-time optimal control with arbitraryfeature-based object recognition and tracking will be thesubject of future work.

ACKNOWLEDGMENT

The authors take this opportunity to acknowledgeSubhransu Mishra for the design and manufacture of the arm.

REFERENCES

[1] Airobots. http://airobots.ing.unibo.it.[2] ARCAS. http://www.arcas-project.eu.[3] D. P. Bertsekas. Nonlinear Programming, 2nd ed. Athena Scientific,

Belmont, MA, 2003.[4] Francesco Bullo and Andrew Lewis. Geometric Control of Mechanical

Systems. Springer, 2004.[5] Roy Featherstone. Rigid Body Dynamics Algorithms. Springer, 2008.[6] D. Fusata, G. Guglieri, and R. Celi. Flight dynamics of an articulated

rotor helicopter with an external slung load. Journal of the AmericanHelicopter Society, 46(1):3–14, 2001.

Page 6: Towards Model-predictive Control for Aerial Pick-and · PDF fileTowards Model-predictive Control for Aerial Pick-and-Place ... on the development and experimental validation of a nonlinear

Fig. 4. Series of pictures showing approach, grasping, and retrieval of the object under NMPC-based reference trajectory setting

[7] M. Hehn and R. D’Andrea. A flying inverted pendulum. In IEEEInternational Conference on Robotics and Automation (ICRA), 2011,pages 763 –770, May 2011.

[8] G. Heredia, A.E. Jimenez-Cano, I. Sanchez, D. Llorente, V. Vega,J. Braga, J.A. Acosta, and A. Ollero. Control of a multirotor outdooraerial manipulator. In International Conference on Intelligent Robotsand Systems (IROS 2014), 2014 IEEE/RSJ, pages 3417–3422, Sept2014.

[9] David H. Jacobson and David Q. Mayne. Differential dynamicprogramming. Modern analytic and computational methods in scienceand mathematics. Elsevier, New York, 1970.

[10] Abhinandan Jain. Robot and Multibody Dynamics: Analysis andAlgorithms. Springer, 2011.

[11] SH Jeong and S Jung. Bilateral teleoperation control of a quadrotorsystem with a haptic device: Experimental studies. 2014.

[12] A.E. Jimenez-Cano, J. Martin, G. Heredia, A. Ollero, and R. Cano.Control of an aerial robot with multi-link arm for assembly tasks. InRobotics and Automation (ICRA), 2013 IEEE International Conferenceon, pages 4916–4921, May 2013.

[13] M. Kobilarov. Discrete optimal control on Lie groups and applicationsto robotic vehicles. In IEEE International Conference on Robotics andAutomation, pages 5523–5529, 2014.

[14] M. Kobilarov. Nonlinear trajectory control of multi-body aerialmanipulators. Journal of Intelligent & Robotic Systems, 73(1-4):679–692, 2014.

[15] M. Kobilarov and J.E. Marsden. Discrete geometric optimal control onLie groups. IEEE Transactions on Robotics,, 27(4):641–655, August2011.

[16] C. Korpela, M. Orsag, T. Danko, B. Kobe, C. McNeil, R. Pisch, andP. Oh. Flight stability in aerial redundant manipulators. In IEEEInternational Conference on Robotics and Automation (ICRA), 2012,pages 3529 –3530, may 2012.

[17] C.M. Korpela, T.W. Danko, and P.Y. Oh. Designing a system formobile manipulation from an unmanned aerial vehicle. In Technologiesfor Practical Robot Applications (TePRA), 2011 IEEE Conference on,pages 109 –114, april 2011.

[18] Quentin J. Lindsey, Daniel Mellinger, and Vijay Kumar. Constructionof cubic structures with quadrotor teams. Robotics: Science andSystems, June 2011.

[19] Vincenzo Lippiello and Fabio Ruggiero. Exploiting redundancy incartesian impedance control of uavs equipped with a robotic arm. InIEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), 2012, pages 3768–3773. IEEE, 2012.

[20] I. Maza, K. Kondak, M. Bernard, and A. Ollero. Multi-uav cooperation

and control for load transportation and deployment. Journal ofIntelligent and Robotic Systems, 57(1-4):417–449, 2010.

[21] Lorenz Meier. Pixhawk autopilot - px4 autopilot platform, 2014.[22] A.Y. Mersha, S. Stramigioli, and R. Carloni. Bilateral teleoperation of

underactuated unmanned aerial vehicles: The virtual slave concept. InIEEE International Conference on Robotics and Automation(ICRA),pages 4614–4620, 2012.

[23] Richard M. Murray, Zexiang Li, and S. Shankar Sastry. A Mathemat-ical Introduction to Robotic Manipulation. CRC, 1994.

[24] M. Orsag, C. Korpela, M. Pekala, and P. Oh. Stability control in aerialmanipulation. In American Control Conference (ACC), 2013, pages5581–5586, 2013.

[25] Matko Orsag, Christopher Korpela, and Paul Oh. Modeling and controlof mm-uav: Mobile manipulating unmanned aerial vehicle. Journal ofIntelligent and Robotic Systems, 69(1-4):227–240, 2013.

[26] P. E. I. Pounds, D. R. Bersak, and A. M. Dollar. Stability of small-scale uav helicopters and quadrotors with added payload mass underpid control. Autonomous Robots, 33(1-2):129–142, 2012.

[27] P.E.I. Pounds, D.R. Bersak, and A.M. Dollar. The yale aerialmanipulator: Grasping in flight. In IEEE International Conferenceon Robotics and Automation (ICRA), 2011, pages 2974 –2975, May2011.

[28] F. Ruggiero, J. Cacace, H. Sadeghian, and V. Lippiello. Impedancecontrol of vtol uavs with a momentum-based external generalizedforces estimator. In IEEE International Conference on Robotics andAutomation (ICRA), 2014, pages 2093–2099, May 2014.

[29] J. Thomas, G. Loianno, K. Sreenath, and V. Kumar. Toward imagebased visual servoing for aerial grasping and perching. In IEEEInternational Conference on Robotics and Automation (ICRA), 2014,pages 2113–2118, May 2014.

[30] Justin Thomas, Joe Polin, Koushil Sreenath, and Vijay Kumar. Avian-inspired grasping for quadrotor micro uavs. In American Society ofMechanical Engineers(ASME) 2013 International Design EngineeringTechnical Conferences and Computers and Information in EngineeringConference, pages V06AT07A014–V06AT07A014. American Societyof Mechanical Engineers, 2013.

[31] A. Torre, D. Mengoli, R. Naldi, F. Forte, A. Macchelli, and L. Marconi.A prototype of aerial manipulator. In IEEE International Conferenceon Intelligent Robots and Systems, pages 2653–2654, 2012.