claraty: coupled layer architecture for robotic autonomy & claraty on fido issa a.d. nesnas ames...

CLARAty: Coupled Layer Architecture for Robotic Autonomy

& CLARAty on FIDO

Issa A.D. Nesnas

Ames Research CenterCarnegie Mellon UniversityJet Propulsion Laboratory

October 25, 2002Mars Technology Program Year-End Review FY02

October 25, 2002 CLARAty FY02 Review- I.A.N. 2

Presentation Overview

• Quad Chart

• Background and Overview

• Relevance to the missions

• CLARAty Team and Collaborators

• Level I & II Milestones

• Accomplishments– CLARAty running on FIDO

– CLARAty Software Package developments

– Distributed development process and testbed

• Future Plans

• Publications and website


Objective Develop a unified and reusable robot control software that facilitates the integration of new technologies on various robotic and rovers platforms

Milestones – FY01 through FY05• Prototype robotic software and implement on

various rover platforms• Integrate and demonstrate technology components

from MTP RMSA program• Demonstrate low-, mid- and high-level autonomy

CLARAty: Coupled Layer Architecture for Robotic Autonomy

Schedule and FundingTask Manager: Issa A.D. Nesnas, JPL(818) 354-9709, nesnas@ jpl.nasa.gov

Participating Organizations: JPLNASA Ames Research CenterCarnegie Mellon University

Facilities:Rocky 8, Rocky 7, K9, FIDO, and CLARAty testbedROAMS, WITSJPL Mars Yard http://claraty.jpl.nasa.gov/

Activity FY00 FY01 FY02 FY03 FY04 FY05

Design CLARAty and implement path planning in it.

Basic rover functionality integrated with decision capabilities

Sophisticated rover functionality for Long Traverses

Sophisticated rover functionality for Instrument Placement

Autonoumous science and Health monitoring


Background & Objectives

Why are we doing this work?• To capture and preserve robotics expertise from

JPL and other centers• To provide a framework for future NASA rover

technology development and integration• To reduce the cost of integrating new technologies• To operate various robots from a unified

framework• To eliminate cost of redeveloping basic robotic

infrastructure


What is CLARAty?

CLARAty is a unified and reusable software that provides robotic functionality and simplifies the integration of new technologies on robotic platforms


Relevance to the Missions

Why is this work relevant to the missions?

• Provides a common environment for development, test, and comparison of advanced robotic technologies

• Provides an infusion path for robotics technologies into flight missions

• Demonstrates technologies on relevant robotic systems

• Makes research rovers viable test platforms for flight algorithms (e.g. navigation)

• Is robust to changes in rover hardware designs

• Can be easily adapted to new research and flight prototype rovers (for ground-based testing)


Measuring Success or Failure

We succeed IF we:

• Significantly reduce integration time of new technology software onto real robotic systems

• Support multiple platforms with different hardware architectures

• Provide a service that is enabling for technologists

• Simplify the development/integrate/debug/test cycle for current and next generation NASA rovers

• Have people other than the developers using and “like” the system


CLARAty Team

CLARAty/Rocky8/Rocky7– Max Bajracharya (34)– Edward Barlow (34)– Caroline Chouinard (36)– Gene Chalfant (34)– Hari Das (34)– Tara Estlin (36)– Dan Gaines (36)– Mehran Gangianpour (34)– Dan Helmick (34)– Won Soo Kim (34)– Michael Mossey (31)– Issa A.D. Nesnas (34) (Task Manager)– Ashitey Trebi-Ollennu (35/34)– Richard Petras (34) (Rocky lead)– Stergios Roumeliotis (Univ. of Minnesota)– Kevin Watson (34)

• NASA Ames Research Center– Maria Bualat– Clay Kunz– Randy Sargent– Anne Wright (lead)

• Carnegie Mellon University– Kam Lasater

– Reid Simmons (lead)

– Chris Urmson

– David Wettergreen– Andy Yang

• Jet Propulsion Laboratory CLARAty on FIDO Team

– Hrand Aghazarian (35/34)– Terrance Huntsberger (35/34)– Chris Leger (35/34)– Matthew Robinson (35/34)

Five Summer Students


CLARAty Collaborations (1/2)

• JPL - MDS Team– Participated with MDS team on CLARAty/MDS infusion task– Supported MDS by upgrading Rocky 7 to MSL-like hardware and

provided working version of CLARAty software for migration to MDS

• JPL - ROAMS Team (A. Jain)– Close collaborations for the integration of the Functional and Decision

Layers to the ROAM rover simulation

• JPL - CLEaR Team (F. Fisher)– Provided an instantiation of the CLARAty Decision Layer, integrated,

tested and debugged on Rocky 7 and Rocky 8 rovers

• JPL - Instrument Simulation Team (M. Lee)– Worked closely with instrument simulation team to develop spectrometer

and spectra infrastructure for CLARAty.


CLARAty Collaborations (2/2)

• RMSA Teams:– ARC - Science Analysis Team (T. Roush)

• Close interactions to support integration of science analysis into the CLARAty environment.

– JPL - Slope Navigation (L. Matthies)• Collaboration for integration of visual odometry on rover platforms

– University of Washington (C. Olson & R. Li)• Close collaboration for integration of bundle adjustment and wide

baseline stereo– Carnegie Mellon University (A. Stentz)

• Integration of D* path planner for long range traverse– Ames Research Center (E. Bandari)

• Integration of visual tracking

• ALERT Team (former REE)– Providing CLARAty communication and rover control infra-structure to

REE for a navigation demonstration on Rocky 8


• Level I: Demonstrate long range traverse using the integration of global path planning and local navigation capabilities provided through the CLARAty architecture (09/2002)

• Level I: Demonstrate a CLARAty capability such as vision-based navigation (using the GESTALT navigator) onto the FIDO rover platform

• Level II: Demonstrate mission-like scenario for visiting multiple science targets using improved position estimation

• Demonstrate CLARAty locomotion functionality by interfacing to ROAMS at the motor control

Milestones for FY02

Accomplishments

THE DECISION LAYER: Declarative model-based Mission and system constraintsGlobal planning

CLARAty = Coupled Layer Architecture for Robotic Autonomy

INTERFACE:

Access to various levelsCommanding and updates

THE FUNCTIONAL LAYER: Object-oriented abstractionsAutonomous behaviorBasic system functionality

A Two-Layered Architecture

Adaptation to a system


General

• ITAR document complete and submitted to NASA HQ for submission to State Department

• Setup several working groups with flexible signup• Upgraded and consolidate all VxWorks licenses (6 seats

with full tool suite support)• Setup a repository for 3rd software CLARAty depends on

(ACE, Perl, Qt, etc.)• Multiple team members can administer CLARAty site• Established web-based operations (testbed, meetings, plans,

procurements)• CLARAty moved to using ACE for OS portability• CLARAty supports several systems: VxWorks, Linux,

Solaris, Mac X


Solaris

x86Ames

JPL

Rocky 8x86

VxWorks

Linux

Rocky 7

K9

ROAMS

Currently Supported Platforms

FIDO

CMULinux

ATRVx86

JPL

FIDOVxWorks

x86

JPL

ppcVxWorks

JPL

Linux

Challenges in Interoperability


Distributed Hardware Architecture

x86 ArchWireless ethernet1394 FireWireI2C Bus

Rocky 8

Compact PCI

WidgetsSingle Axis ControllersCurrent SensingDigital I/OAnalog I/O

Actuator/Encoders Potentiometers

I2C Serial Bus

1394 Bus

IMU

RS232 Serial

Sun Sensor


Custom Architecture/Variability

m68k ArchFramegrabbersDigital I/OAnalog I/OWireless ethernetRocky 7

VME Arch

Actuator/Encoders

Potentiometers

Parallel Custom InterfaceMUX/Handshaking

Video Switcher

GyrosAccels

AIO

AIO

PID Controllers


Centralized Hardware Mapped Architecture

x86 ArchFramegrabbersDigital I/OAnalog I/OWireless ethernetFido

PC104

Actuator/Encoders

Potentiometers

PID Control in Software

Video Switcher

IMU

RS232 Serial


CLARAty on FIDO Adaptation

Controlled_Motor_Impl

ControlledMotor

Linear_AxisJoint

Non-Resuable Layer R8_MotorFido_Motor Sim_Motor

Trajectory

Trajectory_Generator

Mz<Type>

HCTL_Chip

Widget_Board

Widget_Motor

PID ControllerCounter

DIO

Analog Out Analog In

MSI P460

MSI P430 MSI P415

MSI P430

Non-ResuableResuable


Rocky 7 ModulesLines of Code Status

Controlled Motor 2,652 Reusable

Input Output 2,690 Reusable

Bits 1,580 Reusable

Resources (Timers, etc) 725 Reusable

Rocky 7 Motor 927 Non-reusable

Rocky 7 H/W Maps 841 Non-reusable

Motor Controlller LM629 1,143 Reusable - LM629

Digital I/O Board (S720) 576 Reusable - S720PCI Components 329 Reusable - PCI

Total 11,463

Total Reusable 85%Total Reusable - Strict 67%

Code Reusability

Rocky 8 ModulesLines of Code Status


Trajectory Generator 691 Reusable


Bits 1,580 Reusable

Resources (Timers, etc.) 725 Reusable

Bits 756 Reusable

Rocky 8 Motor 1,180 Non-reusable

Rocky 8 H/W Maps 626 Non-reusable

Widget Board Software 2,126 Reusable - widget

Motor Controller HCTL 900 Reusable - HCTL

I2C Master 1,165 Reusable - I2CI2C Master Tracii 1,223 Reusable - Tracii

Total 16,314


FIDO ModulesLines of Code Status


Trajectory Generator 691 Reusable

PID Controller 997 Reusable


Bits 1,580 Reusable

Resources (Timers, etc) 725 Reusable

Common Definitions 2,380 Reusable

FIDO Motor 2,086 Non-reusable

FIDO H/W Maps 1,494 Non-reusable

Encoder Counter (ISA P 400) 463 Reusable - H/W

Analog Input Board (MSI P415) 519 Reusable - H/W

Analog Output Board (MSI P460) 462 Reusable - H/WDigital I/O Board (MSI P560) 602 Reusable - HCTL

Total 17,341



Simulation Hardware Drivers

Adapting to a Rover

Generic

Functional Layer

Rocky 8 Specialized

Classes

&

Objects

Connector

Decision Layer

Multi-level access Connector

Rocky 8

Models/

Heuristics


The Decision Layer

Executives

(e.g. TDL)

General Planners

(e.g. CASPER)Activity Database

Rover Models

FL Interface

Plans


The Functional Layer

Transforms

Motion Control

VisionEstimation

Input/Output

Manipulation

Navigation

Communication

Math

Hardware Drivers

Locomotion

Rover

Behaviors

Path Planning

Rocky 8

Rocky 7

K9

FIDO

Science

Simulation

Sensor

Adaptations

Base

Thickness indicates workdone in area in FY02Primarily inCLARAty on FIDO

CLARAty Software Packages


Base Package

Developed Generic Physical Component Base classes• Math and Data Structure modules

– Merging of array/matrix development in Ames branch with the main branch. Added support for sub-arrays, and STL iterators

• Bits and I/O modules (JPL)– New implementation of bits, digital and analog I/O modules to be more consistent

with the rest of CLARAty, to provide support for reentrancy and to increase efficiency

• Telemetry module (ARC)– Added telemetry modules to represent data and parameter sets, handles general

serialization and deserialization, provide multi-threaded clients, and increase efficiency

• Device module (ARC)– Provide base classes for all devices such as cameras, arms, masts, locomotors, etc.

Uses telemetry classes and provides generic interfaces for accessing telemetry from any device in a consistent manner

• Power System module (ARC)– Provides information on devices such as power sources, batteries, and battery

chargers. Implemented specializations for K9 power hardware


• Developed algorithms for partially-steered and fully-steered vehicles

• Developed continuous driving capability (JPL)

• Demonstrated on Rocky8 & Rocky7, and currently on FIDO (JPL)

• Third generation redesign (CMU)– Separated locomotor model from

control – Added concept for wheel, steerable

wheel, drive cmds, and drive sequences.

– Adapting to Rocky 8, Rocky7, ATRV, FIDO, and K9

– Tested on Rocky 8, Rocky 7 and simulation (including ROAMS)

Locomotion Package

Front

x

y zC

(d)Partially Steerable

(e.g. Rocky 7)

Front

x

y zC

(e)All wheel steering

(e.g. Rocky8, Fido, K9)

(c)Steerable Axle (e.g.Hyperion)

Continuous Driving and Separated Model from Control


Example: collaborative development for locomotor

JPL - 1998

CMU - 2002Future

ARC - 2003

• Designed for Rocky 7 • Used Motor class• Separated wheel

control from locomotion• Built-in pose estimation

• Generalized design for wheeled locomotors

• Full and partially steerable vehicle• Used generic motor classes• Implements fixed axle model• Developed continuous driving• Adapted to Rocky 8, Rocky 7, and Sim

Version 1.0 Version 2.0

• Separated model from control• Add separate locomotor state• Add concept of wheel and

steerable wheel, Drive Cmd, Drive Sequence

• Adapt to ATRV, Sim, Rocky 7, Rocky 8

Version 3.0

Redesign/ mature

Redesign/ mature Version 4.0

• Use device and telemetryinfrastructureAdd adaptation to K9

Add

JPL - 2001


Vision Package

• Developed generic infrastructure for vision– Generic Camera, Stereo Camera, Camera Models, Image, Image

operations (e.g. rectification, edge and corner detection), Image I/O, and Image transport

• Provided wrappers for vision algorithms– JPL Stereo, CAHVOR models, ARC Stereo, SVS Stereo, SLOG tracker

• Adapted package to various rovers– Rocky 8, K9, FIDO, Rocky 7 are currently using the same vision

infrastucture

• Worked with Machine Vision Group to support integration visual odometry into CLARAty

• Worked with RMSA team (Olson, Roush), ARC, CMU, M. Lee in the development of this package

Developed generic Image and Camera infrastructure


• Completed and tested FIDO-based EKF port to CLARAty

(JPL) – Two tier Kalman Filter

– Estimating heading (IMU + odometry), x, and y (odometry)

• Created second generation design for the Estimator module (JPL & Univ. of Minnesota)– Handle non-EKF based estimators

– Designed to better integrate with Locomotor and other model infrastructure

– Assume filter primary functions are to propagate state and update measurements

Estimation Package

FIDO EKF now fully integrated and tested in CLARAty


Navigation & Mapper Package

• New modular design for the Navigator (CMU)– Based on generic action-selector - can be adapted to support

navigation technologies– Specialized action selector to Map-based selector– Tightly integrated local and global cost functions (D*)– Navigator interfaces with Locomotor, Location Estimator, and

Terrain Map generator.

• Refactored Gestalt to separate terrain evaluation from action selection (JPL)

• Developed infrastructure to process multi-tiered panoramic images for terrain evaluation (JPL)

• Acquired complete 3-D map of Mars Yard

Developed Generic Navigation Infrastructure


Demonstration of CLARAty Navigation Infrastructure in Simulation (CMU)

DStar Map (final state)

global cost functionMorphin Terrain Analysis

(scene from simulator)


• Developed adaptation of the CLARAty

controlled motor classes to interface with ROAMS simulation (JPL)

• Tested Locomotor sending commands to wheels which in, turn, send commands to controlled motors (JPL)

• Specialized Sim_Motors turn commands into streams for socket-based communication

• Drove Simulated Rocky 8 rover with new locomotor (CMU)

• Interface between DL/FL (rover-level and ROAMS) is also available

Simulation Package

CLARAty Locomotor talking to ROAMS via motor cmds


• Continued interactions with T. Roush’s

team for integration of science analysis and spectrometer functionality in CLARAty

• Analysis code checked in the repository:– Carbonate Analysis– Edge Layer detection

• Continued interactions with Meemong Lee’s for interface with science instruments, analysis, and simulation. Code available in repository– Calibrated Reflectance Spectrum– Uncalibrated Spectrum

Science Package

Infrastructure for Spectrometer and terrain analysis


Decision Layer Package

• Integrated D* Path Planner with Decision Layer

• Developed infrastructure for sharing terrain map with FL

• Migrating a version of CLEaR (CASPER/TDL) into CLARAty repository. Now available are:– TCM - Task Control Management used by TDL

– Tangent Graph Path Planner (as used by DL)

– D* Path Planner (as used by DL)

– TBD: TDL and Java utilities

• Unifying DL dependencies to use the CLARAty modules (e.g. path planners - tangent graph, D*), etc.

• Ported Decision Layer (ASPEN/CASPER) to Linux

Decision Layer integrated D* Path Planner

October 25, 2002 CLARAty FY02 Review- I.A.N. 37Nesnas/JPL

DECISION LAYER

FU

NC

TIO

NA

L

LA

YE

RTerrain Map from Laser

D* Path PlannerTerrain Map from Laser

Path

Rover

Navigator

Position Estimator

Estimator

Kalman Filter

Locomotor

IMU

R7_Rover

R8_Rover

R7_Locomotor

R8_Locomotor

MotorR7_Motor

R8_MotorWidget Board

Stereo Engine

Camera CameraPX610

Camera1394

FUNCTIONAL LAYER

WheelLocomotor Model

Camera_Image

D* Star

JPL Stereo

Connector

CLARAty Level I Milestone


Mars YardExtending the Trailer Garage Door

Rocky 7 going into trailer

Trailer

Working Desk w/ Computer

The unliftable Rocky 8 Main Door

Sand Box (important for indoor testing)

Wood to prevent sand from entering working area

Start Waypoint 1

Waypoint 2

Waypoint 3

Long Range Traverse Scenario

CLARAty Development Process and Testbeds


Repository

Software Development Process

AFS Backbone

CMU JPL

CLARAty

Repository 3rd PartyReleases Web

0

5

10

15

20

25

30

35

40

45

FY00 FY01 FY02

Users/Collaborators

ExtendedTeam

JPL Team

UW...

VxWorks

ARC

Rocky 8

FIDO

Rocky 7

BenchtopsBenchtopsBenchtops

K9

Repository

ATRV

Some CLARAty Statistics~170 Modules (reusable entity)~31 Packages (module grps)~3 rovers ~250,000 lines of C++ code ~Java/scripts/ and models

Authentication


FIDO Benchtop

Rocky 8 Benchtop

Dexter ManipulatorsRocky 7 Benchtop

CLARAty Testbed (1/2)


CLARAty Testbed (2/2)

• 5 cPCI embedded targets (x86, ppc)

• 2 Linux and 5 Solaris hosts/targets

• AFS VxWorks installations (Tornado I, II, II.2, x86, ppc)

• Mockups for Rocky 8, FIDO, and Rocky 7

• Remotely accessible to CLARAty developers & users

• Web-based target status and control

• Small lab sandbox built for indoor testing

• Several remote and local users exercising testbed (MDS, CMU, ARC, JPL)

• Online sign up for target/rover usage

http://claraty.jpl.nasa.gov/testbed


Rocky 8 Upgrades

• Computational modules– CPU up to 1.2 GHz Pentium Pro w/ 256 MB (easily swappable)

– FireWire 1394 Cameras (front/back)

– Wireless 22 MB/sec sysLink Ethernet

– I2C Master Control Board (Tracii)

– Explored upgrade of widget processors to PIC18xx - product has bugs, awaiting new revision

• Electromechanical – Upgraded all motors to COTS motors and encoders

– Rewired motor assemblies to make identical and simplify repair

– New Li-ion battery system (6-7 hour operation)

– Fixed steering slippage problem, upgraded camera and battery boxes.


Rocky 7 Upgrades

• Computational modules– Moved from VME backplane to cPCI

– CPU - PPC750 300MHz w/ 256 MB - extended temperature

– PX610 B/W Imagenation framegrabbers (x2)

– New Digital and Analog I/O boards (S720 and industry pack - Acromag)

– New 11 MB/s sysLink wireless ethernet

• Electromechanical – Fabricated new LM629 custom motion control boards (x3)

– Several mechanical repairs

• Software– Upgraded most CLARAty drivers to support new hardware -

relatively seamless transition


Summary of Accomplishments

• Established a distributed inter-center software development process

• Provided a remotely accessible rover testbed• Collaborated with MDS to define CL/MDS infusion process• Developed major software infrastructure in the following

packages: – Vision, Navigation, Estimation, Science, Locomotion, Motion

Control, I/O, Simulation and FIDO adaptation

• Upgraded Rocky 7 to MSL-like hardware (PPC)• Upgraded Rocky 8 to FireWire cameras • Integrated Decision Layer and path planning• Completed and submitted ITAR document


FY03 PlansDate

ReadyNeeded

By Technology Description Receives FromDate

Needed

01-Dec-02 CLARAty/MDS, LT, IP

CLARAty testbed: Rocky 8, Rocky7, six embedded mockups for R8, R7, FIDO

01-Dec-02 LT, IP CLARAty/ROAMS open-loop locomotion with simulation of closed-loop wheel control

ROAMS with wheel control simulation

01-Nov-02

15-Dec-02 LT EKF position estimation for (x,y, heading)15-Jan-03 LT, IP Visual Odometry on Rocky 8 and Rocky 7 RMSA - Slope Navigation 01-Sep-0231-Jan-03 LT Sun sensor heading estimation FIDO - sun sensor code - legacy

Rocky 7 - sun sensor code - legacy15-Nov-02

01-Feb-03 IP 2D Visual Tracker with obstacle avoidance on Rocky 8 and Rocky 7

RMSA - Visual Tracking 01-Nov-02

15-Feb-03 LT ROAMS/CLARAty with reasonable R8 data (kinematic, mass inertias, etc) with locomotion, wheel control, IMU, and pose estimation

ROAMS version with IMU simulation 01-Dec-02

01-Mar-03 IP GESTALT navigator on Rocky 8 and Rocky 7 (encapsulated latest MER version)

MER - GESTALT navigation software 01-Dec-02

15-Mar-03 CLARAty/WITS - demonstrate uplink and downlink of a single command (e.g. Hazcam)

WITS version to support interface 01-Feb-03

01-Apr-03 LT, IP Two navigators designed with CLARAty infrastucture - Morphin++, Gestalt

Morphin++ - navigation softwareMER - GESTALT navigation software

01-Dec-02

15-May-03 IP Stereo-based manipulation infrastructure - control/coordination/sensing for Rocky 8, PDM, and K9 Arms and masts

PDM - legacy instrument placement

01-Jun-03 LT CLARAty/WITS - demonstrate commanding for locomotion (driving) and navigation

15-Jun-03 IP CLARAty/ROAMS interface for rover-based manipulators

ROAMS simulation of manipulator kinematics and interface

01-Apr-03

30-Jun-03 IP CLARAty/WITS - demonstrate manipulator commanding for Rocky 8Define format for shared models

WITS version to support Rocky 8 commanding interface

01-Mar-03


Publications & Website

http://claraty.jpl.nasa.gov

• T. Estlin, F. Fisher, D. Gaines, C. Chouinard, S. Schaffer, and I. Nesnas, “Continuous Planning and Execution for an Autonomous Rover,” To appear in the Proceedings of the Third International Workshop on Planning and Scheduling for Space, Houston, TX, Oct 2002

• I.A.D. Nesnas, “CLARAty - An Architecture for Reusable Robotic Software,” CSMISS IT Spotlight Series, June 19, 2002

• I.A.D. Nesnas, R. Volpe, T. Estlin, H. Das, R. Petras D. Mutz, "Toward Developing Reusable Software Components for Robotic Applications" Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Maui Hawaii, Oct. 29 - Nov. 3 2001. pdf (8 pages, 2MB)

• T. Estlin, R. Volpe, I.A.D. Nesnas, D. Mutz, F. Fisher, B. Engelhardt, S. Chien, "Decision-Making in a Robotic Architecture for Autonomy." Proceedings of 6th International Symposium on Artificial Intelligence, Robotics, and Automation in Space (i-SAIRAS), Montreal Canada, June 18-21 2001. pdf (8 pages, 72KB)

• R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "The CLARAty Architecture for Robotic Autonomy." Proceedings of the 2001 IEEE Aerospace Conference, Big Sky Montana, March 10-17 2001. pdf (12 pages, 470 KB)

• R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "CLARAty: Coupled Layer Architecture for Robotic Autonomy." JPL Technical Report D-19975, Dec 2000. pdf (116 pages, 904 KB)

Thank you for your Attention

claraty: coupled layer architecture for robotic autonomy & claraty on fido issa a.d. nesnas ames...

Documents

robotic functionality

robotic autonomy claraty

robotic autonomyschedule

fy05prototype robotic

websiteclaraty fy02

systemclaraty fy02 review

new research

claraty testbedroams