feifarek 1 mapld 2005/a220 jonathan f. feifarek [email protected] timothy c. gallagher...

Feifarek 1 MAPLD 2005/A220

Jonathan F. [email protected]

Timothy C. [email protected]

Lockheed Martin Space Systems Co.

FPGA Based Processor FPGA Based Processor for Hubble Space Telescopefor Hubble Space Telescope

Autonomous Docking – A Case Study Autonomous Docking – A Case Study

Courtesy NASA GSFC


Background: Need for Hubble Repair

● 4 / 1990: Hubble Space Telescope (HST) launch

● 12/1993: SM* 1- Corrective COSTAR, WFP Camera2

● 2 / 1997: SM 2 – Add NICMOS, STIS, Thermal Blankets

● 10/1997: Hubble Operations Extended from 2005 to 2010

● 12/1997: SM3A Replace 6 Gyros, 3 Fine Guidance Sensors

● 3 / 2002: SM3B Replace Solar Panels, NICMOS Coolant

● 3 / 2003: SM 4 Cancelled Following Columbia Disaster

● 6 / 2004: Hubble HRV Request For Proposal Issued

● 8 / 2004: Lockheed Martin awarded HST Robotic Vehicle (HRV)

● 12/2007: Target HRV Launch Date * SM = Service Mission


HRV Mission : Autonomous Docking

Mission Phase System ApproachRequirement

Pursuit

• Orbit phasing with HST

• HRV checkout

• Range from HST for initial sensor acquisition

Proximity Ops

• HST approach with safe-hold points

• Acquire sensor data on HST orientation and rotation rate

Approach

Capture/Berth

• Rate matching with HST

• Maneuver to HST capture point

• Capture HST:

– Robotic Arm Captures HST Grapple Fixture

– Berth to HST aft interface


Vision Processing Algorithm Selection Criteria● Implementation Concerns - Computational Intensive

– Field Programmable Gate Array (FPGA)– Flight Computer, DSP Processor– Combination

● Implementation Approach– All: Conventional Programming Languages– FPGA High-Order Languages (HOLs)– FPGA Register Transfer Logic (RTL) in VHDL or Verilog

● Error-prone● Time consuming (calendar time plus engineering cost)● Difficult to achieve bit accurate & cycle accurate

operations using hand-coded conversions


Vision Processing Algorithm Selection Results

● FPGA Reconfigurable Architecture Chosen– Searched Internet and Conference Proceedings for comparisons

between Processors and FPGA Reconfigurable Computer (RCC)● Space Based RCC technology leaders such as Los Alamos

National Labs1 and NASA2 noted FPGA system performed between 10-1000x faster then processors

● Many other references on FPGA based accelerated image processing from University studies3,4

● Microprocessor Embedded in FPGA– Allows rapid evaluation of architecture performance– Can host large amounts of existing code such as decision logic

and complex sequential math– For certain algorithms Floating Point is more efficiently

implemented in processor code then in gates


Vision Processing Algorithm Selection Results

● FPGA Implementation: Combination of HOL, RTL– HOL (Celoxica Handel-C) for fast and efficient implementation– Provided fast development cycle needed for mission

● Quickly ported math libraries & existing C++ code● Performance matched RTL speed, area ; slower than hand-

code ● Highest speed increase from hand floorplanning

– RTL for IO Wrapper, IO reuse, and custom-optimized code– Combined the benefits of all worlds– Microprocessor Implementation

● Incorporated Xilinx MicroBlaze ™ Core in FPGA ● Xilinx tools: Platform Studio© SDK / EDK suite● Used gnu© C compiler / “gdb” debugger


Vision Processing FPGA Development Flow

C Algorithm Acceleration C to RTLGenerate human-readable VHDL and Verilog for 3rd party synthesis

C to FPGADirect implementation to device optimized programmable logic

FPGA

ImplementationProvide rapid iteration of partitioning decisions throughout flow

VerificationDrive continuous system verification from concept to hardware

Used with permission of Celoxica, Inc.


Vision Processing Card (VPC) Block Diagram

Program,Data Memory

Raw images Edge Enhanced Images

Xilinx microBlaze™MicroProcessor Core

Image Points

Edge Finder

Image Patches

Front End Image

Processor

Pyramidal Downsampling /

Edge Enhancement

Serial Camera Pixels

Project Model Points(3D to 2D Images)

Compute New Pose

Project Edges

Compute New Pose (Iterative)

Output Pose

RAM

Loadingt+1

Lukas Kanade Trackers

Image Points

Memory ManagerMemory Manager

Enhanced PixelsPixels

t Loading t

DataFloating Point Unit

Pipeline

Custom Floating Point Unit (FPU)

Data

Operator

Control

Scaler,Matrix

* / -Convert

Operation Request

Operands In,Results Out

Edge

Best Fit Edge

uBlaze Software Libs

uBlaze Hardware FPU

Single FPU Instance With Multiple Software Invocations

uBlazeIn/Out

FPGA


Vision Processor Card Architecture

SRAM

Flash

PowerSwitch

PCI-PCIBridge / Config

Internal PCI

COP AXilinx V2

Port AS

RA

M

SD

RA

M

COP BXilinx V2

Port B

COP CXilinx V2

Port C

COP DXilinx V2

Port D

Common Interconnect Bus

J8PCI Connectors

SerDesS

DR

AM

SR

AM

SD

RA

M

SerDes

SD

RA

M

SR

AM

SD

RA

M

SerDes

SD

RA

M

SR

AM

SD

RA

M

SerDes

SD

RA

M


VPC Engineering Development Board

Used with permission of SEAKR Engineering, Inc.


VPC SEU Approach

● Main SEU Mitigation: Dual Voting at FPGA output

– Detects SEE's but cannot correct for them

– Tight power restrictions (thermal reasons) restrict triple voting

– Vision Processing Algorithm tolerant of drop-outs

● Multiple camera views / algorithms into Kahlman filter● HRV mission uses very low rate docking (1 inch / sec)

● SEU Correction at FPGA-to-Memory Interfaces

● Microprocessor returned to Reset State after each image

● Algorithm memory only 1 image deep; flushes SEU effects

● Voting, Configuration Scrubbing Performed in Rad Hard Part

● Analysis Shows Low SEE Rate (1 effective upset / 10 hours)


VPC Sizing Results for NFIR Algorithm

LUTs Flip Flops Multipliers BlockRAMsMicroBlaze Processor 2700 2000 4 33Front End + 8000 4000 0 42LK Tracker 7000 2700 19 24Total 17700 8700 23 99Available 67584 67584 144 144Percentage Utilized 26% 13% 16% 69%

LUTs Flip Flops Multipliers BlockRAMsMicroBlaze Processor 2700 2000 4 33Front End + 8000 4000 0 42LK Tracker (4) 13580 10580 70 68Total 24280 16580 74 143Available 67584 67584 144 144Percentage Utilized 36% 25% 51% 99%

Single LK Tracker

Quad LK Trackers


VPC Performance Results for NFIR Algorithm

Function Timed Cycles/Loop Loops Total CyclesProject Model Points 26000 1 26000Lktracker (hardware) 2000000 1 2000000FindExtrinsic 3078000 1 3078000Project edges 120000 3 360000FindEdges (hardware) 400000 1 400000Project ellipses 80000 3 240000computeAllFis 180000 2 360000computeVsumCsum 280000 2 560000computeAlpha 230000 2 460000UpdatePose 6000 3 18000getAllErrors 240000 3 720000

Total 6640000 8222000


VPC Performance Results for NFIR Algorithm (cont.)

FindExtrinsic timing Cycles/Loop Loops Total CyclesNormalize 73000 1 73000SVD6x6 170000 3 510000SVD3x3 35000 1 35000FindHomography 1100000 1 1100000ProjectPoints 190000 4 760000Rest 600000 1 600000

Total 2168000 3078000


Summary: Lessons Learned

● Using OpenGL algorithm for development hampered design

● Parallel PC board and FPGA designs helped meet schedule

● Using FPGA’s was key to meeting speed requirements

● Use of microprocessor core reduced development time

● Early allocation of algorithm to hardware/software paid off

● Use of HOLs made implementing complex tasks possible

● Engage expert tool user on team (MicroBlaze, Handel-C)

● Having reference software / test data eased verification

● Benefited from small, enthusiastic, tight knit team

● Worked around MicroBlaze libraries bugs with custom logic


References

● (1) “A Space Based Reconfigurable Radio”, Michael Caffrey,Los Alamos National Laboratory, MAPLD September 2002

● (2) “Developing Reconfigurable Computing Systems for Space Flight Applications”, Thomas P. Flatley,NASA Goddard Space Flight Center Greenbelt, Maryland 20771

● (3) "Implementing Image Applications on FPGAs," B. Draper, R. Beveridge, W. Böhm, C. Ross and M. Chawathe. International Conference on Pattern Recognition, Quebec City, Aug. 11-15, 2002.

● (4) “Performance of Reconfigurable Architectures for Image-Processing Applications”, Domingo Benitez, University of Las Palmas G.C.,Journal of Systems Architecture: the EUROMICRO Journal, September 2003

feifarek 1 mapld 2005/a220 jonathan f. feifarek [email protected] timothy c. gallagher...

Documents