georgia tech uav research facility: navigation and control ... · june 06 enj - georgia tech -...

Eric N. JohnsonLockheed Martin Assistant Professor of Avionics Integration

Daniel Guggenheim School of Aerospace EngineeringGeorgia Institute of Technology

Seminar at DLR, Braunschweig, GermanyMay 23, 2006

Georgia Tech UAV Research Facility:Navigation and Control Research Highlights

June 06 ENJ - Georgia Tech - uav.ae.gatech.edu 2

Outline

• Adaptive Control– GTMax research UAV– GTSpy small ducted fan– Edge aerobatic research aircraft

• Active Vision– Multi-University Research Initiative (MURI), Active-vision control

systems for complex adversarial 3-D environments– International Aerial Robotics Competition


Neural-Network Adaptive Control

ApproximateDynamicInversion

ApproximateDynamicInversion

Pseudo-Control

νPlantPlant

δ

Plant Inputs (Actual Controls)

+Reference Model

Reference Model

Command

rmν

PDControl

PDControl

NeuralNetworkNeural

Network

-

+ TrackingError

pdνadν−

Doesn’t require high fidelity nonlinear flight dynamics model


( ) azez −+=

11σ

( )xVW TTσyad ==νIn matrix form:In matrix form:

• Feedforward neural networks with a single hidden layer are universal approximators

Single Hidden Layer Neural Network

( )σ ⋅

( )σ ⋅

( )σ ⋅

( )σ ⋅

MM

x1

x2

xN1

M

y1

y2

yN3

V W

N2

N1 N3


Neural Network Adaptive Control

PDControl

PDControl

DynamicInversionDynamicInversion

NeuralNetworkNeural

Network

PlantPlantReference Model

Reference Model

-

++

TrackingError

Command

cmdδ ActuatorActuatorδν

rmν

pdνadν−


Neural Network Adaptive Control with Pseudo-Control Hedging

PDControl

PDControl

DynamicInversionDynamicInversion

NeuralNetworkNeural

Network

PlantPlantReference Model

Reference Model

-

++

TrackingError

CommandEstimateHedge

EstimateHedge

ActuatorActuatorcmdδ

hedgeν

δν

x

rmν

pdνadν−


Implications

• “Shelter” adaptive element from the adverse effects of plant input characteristics: – Linear dynamics, latency, actuator saturation, rate saturation, etc.

• Achievable adaptation performance is increased dramatically

• Adaptation is correct during saturation– Adaptive element can recover from a period of “faulty” adaptation


Georgia Tech R-Max: GTMax

• Yamaha R-Max, – 66kg – 3m Rotor

• Instrumented as a Research VTOL UAV

• More than 300 research test flights since March 2002

• Platform for DARPA Software Enabled Control program rotary wing final experiments and demonstrations in Summer 2004

GTMax Automatic landing during November 2002 SEC Meeting


GTMax Avionics HardwareManual pilot

antenna

Datalink antennae

Vibration isolated avionics rack

GPS antenna

Sonar

• Flight Computers– 266MHz & 800 MHz

embedded PCs, Ethernet, flash drives

• Sensors– Inertial Measurement Units

(x2)– Differential GPS– Magnetometer, sonar

altimeter– RPM, voltage/fuel warnings– Various camera systems

• Data Links– 11 Mbps Ethernet data link– RS-232 serial data link

• Flight Computers– 266MHz & 800 MHz

embedded PCs, Ethernet, flash drives

• Sensors– Inertial Measurement Units

(x2)– Differential GPS– Magnetometer, sonar

altimeter– RPM, voltage/fuel warnings– Various camera systems

• Data Links– 11 Mbps Ethernet data link– RS-232 serial data link


Ground Control StationLaptopsInside

LaptopsInside

Datalink 2Datalink 2

Datalink 1Datalink 1

Ref. GPSRef. GPS


GTMax Avionics Installation and Wiring

Primary Flight Computer

Serial Extension Board

FreewaveDGR-115

DC/DC

5V

5V

Battery 12V

Aux

iliar

yM

odul

e

AironetMC4800

EthernetHub

NovAtel RT-2 GPS Receiver

Secondary Computer(Image Processor)

FCS20 Small Autopilot

ISIS-IMU

12V

DC/DC

DC/DC

HMR-2300Magnetometer

DC/DC

Sonar Altimeter

Power DistributionModule

Generator

Yamaha AttitudeControl System

RC Receiver YACS IMU

12V5V

RS-232 SerialEthernetDC Power

Dat

a Li

nkM

odul

eG

PS M

odul

eFC

S20

GPS

Flig

ht C

ompu

ter

Mod

ule

5V

12V

Pan/tilt/roll hardware(power wiring not shown)

Video Transmitter

Camera(several types)

Servos

EncodersEncoders

Encoders

ServosServos


GTMax Onboard Baseline Software

• Navigation– 17 State Extended Kalman

Filter Navigation System• Position, velocity, attitude• Accelerometer/gyro biases• Terrain height error

– All Attitude Capable– 100 Hz updates

• Control– Adaptive Neural Network

Trajectory Following Controller – Neural Network: 7 Outputs for

7 Degrees of Freedom– Full speed envelope– High performance position

tracking

• Datalink and Comms– Handles all serial devices – Reroute data over Ethernet,

save to file, playback– Software reconfigurable

• Trajectory Generation– Generates smooth

position/velocity/accel/attitude commands as a function of time from waypoints


Neural Network Adaptive Flight Control

• Simultaneous adaptation in 7 degrees of freedom• Address large model errors and disturbances

– GTMax model is based on a linear model for hover - adaptation used on for flight over the entire flight envelope of the helicopter

– Different configurations, damage

• Accounts for actuator saturation and “two-loop” design

• Tested over full speedenvelope, aggressive maneuvers

InnerLoop

External Command

OuterLoop

Online trainedNeural

Network

Attitude corrections


Automatic Takeoff


20 ft. lateral step response

Automatic Rapid Reposition


Automatic Landing


Auto Pirouette


Auto 180o Velocity Change


Automatic Flight at 95 ft/sec

9160 9180 9200 92200

20

40

60

80

Time (sec)

Gro

und

Spe

ed (f

t/sec

)

Upwind Downwind


Many Configurations…


Rotor synchronizes with video at 900RPM

Using RPM to Accommodate Frozen Collective Actuator (10 ft Climb)

10ft


Flying 50 ft square

Frozen “Right” Swash-Plate Actuator


Small Autopilot Development (FCS20)

• 60mm x 90mm x 32mm, 120 grams– DSP for floating point operations (1.3Gflops)– FPGA IO interface– MEMS rate gyros, accelerometers– Air data– GPS

• Same software as baseline GTMax– 17 state extended Kalman filter navigation– Neural network adaptive trajectory-following flight control– Trajectory generator


GTSpy Description

• Modified from MASS Helispy• FCS20 Small autopilot• Weight - 5 lbs• Height - 27 in• Duct Diameter - 11 in• Hover Endurance - 25 min


Autopilot Tests on GTSpy, Takeoff

5 pounds11 inch duct

(movie) http://uav.ae.gatech.edu/videos/s040807k1_takeoff.mpg


Autopilot Tests on GTSpy


(movie) http://uav.ae.gatech.edu/videos/s040807k3_right40ft.mpg


Autopilot Tests on GTSpy, Landing


(movie) http://uav.ae.gatech.edu/videos/s040807k5_landing.mpg


First Air Launching of a Hovering AircraftNovember 20, 2004

• GTMax at 400 feet• GTSpy Launched, then

hovers 80 feet below• Both vehicles utilizing the

same NN adaptive controller software

(movie 1) http://uav.ae.gatech.edu/videos/fs041120c1_launchHeliSpyLong.mpg (movie 2) http://uav.ae.gatech.edu/videos/fs041120c1ob_launchHeliSpy.mpg


Agile Maneuvering Test Aircraft: GTEdge

• 33% scale Edge 540T

• Avionics– FCS20 Small Autopilot– NovAtel DGPS– FreeWave wireless serial

• Flights began June 2005

60 ft/sec Autoflight(movie) http://uav.ae.gatech.edu/videos/e050624b3_60fps.mpg


Automatic Transition to Zero Speed• First transition to near-vertical attitude

and near-zero airspeed, July 2005– Video begins at about 880 sec– Slow level-flight envelope

expansion from forward flight to hover

0

500

1000

-500

0

500

10000

500

1000

1500

North (ft)

trajectory

West (ft)

heig

ht (f

t)

measuredcommandend

800 850 900 9500

20

40

60

time (sec)

spee

d (ft

/sec

)

800 850 900 9500

20

40

60

80

time (sec)

pitc

h an

gle

(deg

)

800 850 900 950-100

-50

0

50

100

time (sec)

elev

ator

(% o

f max

)

800 850 900 9500

20

40

60

80

100

time (sec)

thro

ttle

(%)

800 850 900 950-100

-50

0

50

100

time (sec)

aile

ron

(% o

f max

)

800 850 900 950-100

-50

0

50

100

time (sec)

rudd

er (

% o

f max

)

(movie) http://uav.ae.gatech.edu/videos/e050723c3_vertical.mpgand e050723c2_30fps20fps.mpg


High Speed: 160 ft/sec (95 knots)

(movie) http://uav.ae.gatech.edu/videos/e060128a2_160fps.mpg


Rapid Transition to Hover & Return to Forward Flight

(movie) http://uav.ae.gatech.edu/videos/e060509a1_rapidTransition.mpg


Other Collaborations

AFRL & Aerovironment, Skytote

DARPA & Frontier Systems / Boeing, Maverick/Renegade

D6


Active-Vision Control Systems for Complex Adversarial 3-D Environments• AFOSR/AFRL MURI: Georgia Tech, MIT, UCLA,

Virginia Tech• Development of sound methods that utilize 2-D and

3-D imagery to– Enable aerial vehicles to autonomously detect and prosecute targets in

uncertain complex 3-D adversarial environments– Do these things without relying upon highly accurate 3-D models of the

environment– Include capabilities and approaches inspired by those found in nature

• New strategies of – Target recognition/tracking– Obstacle/hazard avoidance– Navigation, guidance, and control

Active-Vision Control


Tracking with Region-Based Deformotion

• Simultaneous segmentation, registration flow

• Registration using energy minimization with gradient descent

• Handle occlusions


Vision-Only Guidance, Navigation, and Control


The Measurements

Available measurements from a candidate window• Size• Rotation• Horizontal pos. in camera• Vertical pos. in camera


Flight Test Results


Vision-Aided Inertial Navigation

• Inertial Navigation System (INS) aided by 2-D vision sensor looking at a selected image target

Autopilot UAV

IMU + Navigation

Camera Image Processing+Computer Vision


Vision-Aided Inertial Navigation

• Approach– Use assumed range (altitude), position in image, size of object in

image, and aircraft state to estimate object position, size, andorientation

– Use subsequent measurement of object image position and size in image to update INS

– Utilize inertial data to maintain lock on target

– Operate without GPS (or any other position aiding)

Last Object LocationLast Object Location

Actual NewObjectLocation

ExtrapolatedNew Object Location∆z

∆y

Actual NewObjectLocation

ExtrapolatedNew Object Location∆z

∆y


Vision-Aided INS Trials

Special Purpose Optical Target

Using Existing Features(in this case a window)


2GT

Close Approach to Building, no GPS


Flight Test Results: Error Plots

error = |estimate – GPS|

Max Errors:9.6 ft (x)8.3 ft (y)4.0 ft (z)


Air-to-Air Vision-Based Tracking

Guidance Autopilot UAV

IMU + Navigation

Camera

Estimated:LOS RateRangeObject Size

Acc Com

PositionVelocityAccelerationRates

Kalman Filter Image Processing+Computer Vision

No information is communicated between aircraft, and only passive 2-D vision information is available to maintain formation

Camera

Y

X

y

x

Camera Image Plane

Target

Image frame

Z

u = (uX, uY, uZ) : unit vector α : subtended angle

r : range b : target size

Estimation Stateu, u, , , b. 1 r

r r .Estimation State

u, u, , , b. 1 r r r

.u, u, , , b. 1 r

r r u, u, , , b. 1 r r r

.

Measurementu, α


• Fast marching methods + target acquisition process= 10 frames/sec

Video 1 Video 2

Tests on Recorded Videos


Estimator DesignApply Extended Kalman Filter (EKF) to• Measurement

calculated by using image processor outputs

• Estimation state

Unit vector expressed in a camera frame

Unit vector expressed in a NED frame

Leader

Camera frame

Xc

Yc

Zc

North

EastDown

Follower

u : unit vector

α : subtended angle

r : range

leader’ssize : b

alat : leader’s lateral acceleration

Measurement and Estimates

Leader’s lateral accelerationIn the leader’s wind frame( perpendicular to leader’s velocity vector )


6-DOF Image-in-the-Loop Sim Results

• Relative position • Relative velocity

ImageProcessor EKF Controllerz x̂

Simulator


Simulation Results: Image Processor• Image source : recorded video in flight test• Simulated camera motion : circling with a constant speed• Commanded camera position : 100ft behind, 20ft below the leader

• Image processor outputs

Leader’s positions in image Unit vector and subtended angle


Simulation Results: 3D EstimatorPosition

Velocity Acceleration


Flight Test Results• Image processor outputs and EKF estimates

Unit vector and subtended angle

0.94 0.96 0.98 1 1.02 1.04 1.06

x 104

-0.5

0

0.5

1

navTime

u

u1u2u3

0.94 0.96 0.98 1 1.02 1.04 1.06

104

0.05

0.1

0.15

0.2

0.25

navTime

α

Leader position

0.94 0.96 0.98 1 1.02 1.04 1.06

x 104

0

1000

2000

navTime

X

VISIONGPS

0.94 0.96 0.98 1 1.02 1.04 1.06

x 104

0

500

1000

1500

navTimeY

VISIONGPS

0.94 0.96 0.98 1 1.02 1.04 1.06

104

-500

-400

-300

-200

navTime

Z

VISIONGPS


Formation Flight Tests (GTMax/GTEdge)


Vision-Based Formation Flight Tests (GTMax/GTEdge)


Status

• Currently “closing the loop” on guidance –true vision-based formation flight

• First attempts were earlier this week• Longest run 22 seconds• Current efforts include more advanced vision

processing, estimation, and guidance• Moving into other intended scenarios:

see/avoid, pursuit


Box Target Maneuver Sinusoidal Target Maneuver

Neural Network augmenting

estimator corrects bias

}}

Estimating Dynamics of Maneuvering Target


International Aerial Robotics Competition

Launch area

Two lights near building

Vehicle or subvehicle(s) enter building

Building and an Entry Point Found

>1m

Transmit an image of “point of interest”inside building

In less than 15 minutes:

Image receiver(& other ground components)Sign on

building

3 km

http://avdil.gtri.gatech.edu/AUVS/CurrentIARC/2001CollegiateRules.html


Levels

Level 1: Follow prescribed waypoints for 3kmLevel 2: Locate building and find an entry

• Level 3: Operate inside the building – Can be a different vehicle or subvehicle than used above– Can launch near target structure

• Level 4: All of the above– Complete in < 15 minutes

Contest gets new rules once somebody does level 4


AUVSI InternationalAerial Robotics Competition

Launch AreaIdentify building and map entry point usingonboard systems

>1m

Image receiver(For judges)

Sign on correctbuilding

2003 ApproachGoal: Level 2 Mission


Simulation and Flight Tests in Preparation for 2003 Competition


2003 Results, Level 2

• Allowed Four Mission Attempts• On Three of the Attempts:

– Building Located by Recognizing Symbol• 1m x 1m sign• Fifteen buildings to chose from

– Valid Opening Found and Automatically Reported to Judges– Took 16-17 minutes each time


Future Plans

• Continue to expand on air-to-air tracking– Cluttered background– Flight testing and facilitate technology transition

• Air-to-ground tracking (by airplanes)– Leverage air-to-air work (particularly cluttered background and

guidance work)

• Obstacle avoidance– Initial emphasis on unknown fixed obstacles, ownship state known– Using passive monocular 2-D sensor

• Requires considerable work in image processing, 3-D estimation, and guidance


Expected Outcomes and Transitions

• New capabilities of autonomous sensing and control, enabling operations: – In the presence of uncertainty– In a clandestine/covert manner– In close proximity to hazards, structures, and/or terrain

• Relevant flight test validation• Enable more capable/reliable existing air vehicles

and guided munitions• Enable entirely new systems to be developed

(for example, capable of operating in urban environments)


Acknowledgements

• Sponsors: DARPA, AFRL, AFOSR, NSF, NASA, Lockheed Martin• Partners: GST, Boeing, Draper, Lockheed Martin, Northrop Grumman,

Honeywell, McKenna MOUT, SSCI, OGI, Virginia Tech, MIT, UCLA• Georgia Tech Participants:

Thomas Apker, Anthony Arkwright, Mike Baldwin, Sean Barnes, Anthony Calise, Cesar Carrillo, Jane Case, Claus Christmann, Henrik Christophersen, Scott Clements, Mike Curry, Joerg Dittrich, Graham Drozeski, Jason File, Nate Fisher, Lucas Fortier, Stewart Geyer, Richard Giuly, Luis Gutierrez, Bryan Gwin, Jincheol Ha, Greg Ivey, Mat Hart, Bonnie Heck, Jeong Hur, Anna Ivester, Ole Jakobsen, Eric Johnson, Phillip Jones, Suresh Kannan, Adrian Koller, Dan Laszweski, Tom Linden, Sumit Mishra, Alex Moodie, Kyungjin Moon, Hiong Chair Naik, Wayne Pickell, Seung-Min Oh, J.V.R. Prasad, Alison Proctor, Nimrod Rooz, Bhaskar Saha, Daniel Schrage, Liang Tang, Allen Tannenbaum, Mike Turbe, Shannon Twigg, Suraj Unnikrishnan, George Vachtsevanos, Patricio Vela, Yoko Watanabe, Linda Wills, Allen Wu, Ilkay Yavrucuk, and many others…

georgia tech uav research facility: navigation and control ... · june 06 enj - georgia tech -...

Documents