math works
TRANSCRIPT
Design of RealDesign of Real--Time Video Time Video Processing Systems in Processing Systems in Simulink With the Video & Simulink With the Video & Image Processing BlocksetImage Processing BlocksetDavid JacksonProduct Marketing ManagerVideo & Signal ProcessingThe [email protected]
2© 2005 Altera Corporation
AgendaAgenda! Video & Imaging System Design
− Imaging with MATLAB®
− Key Challenges− Model-Based Design− Simulink®
! Featured Product:− Video & Image Processing Blockset
! Executable Specifications− Benefits, ROI Study
! Video Design Example− Edge Detection
! Q&A
3© 2005 Altera Corporation
Intelligent Video DemonstrationIntelligent Video Demonstration! Traffic Sign Detection Example
4© 2005 Altera Corporation
! Headquarters: Natick, Massachusetts, USA! European Offices: UK, France, Germany,
Spain, Benelux, Italy, Switzerland, Nordic! North American Offices: CA, MI,
Washington DC, TX! Asia/Pacific Office: Korea! Worldwide Consulting ! Distributors in 20 Countries
Earth’s Topography on an Equidistant Cylindrical Projection, Created with the MATLAB Mapping Toolbox
The MathWorks at a GlanceThe MathWorks at a Glance
! Revenues of ~$300M in 2004! Privately Held! Over 1,000 Employees Worldwide,
1/3 in Product Development! Worldwide Revenue Balance:
50% North America, 50% International! More than 1,000,000 Users in 175+ Countries
6© 2005 Altera Corporation
DefenseAutomotive Safety
Semiconductor
CommunicationsUAV
Medical DevicesSecurity CamerasOffice Equipment
Some Image & Video Processing SystemsSome Image & Video Processing Systems
7© 2005 Altera Corporation
Video System Design ChallengesVideo System Design Challenges! Extreme Computation Demands! Embedded System
Resource Constraints−Real-Time Requirements
! Designing for a Target Hardware−DSP, FPGA
! End-Product Performance, Power, Size−Roadmap for Adding Features,
Performance
! Testing & Validating Results
8© 2005 Altera Corporation
! Foundation for Model-Based Design, Automatic Code Generation, & Verification & Validation
! Open Architecture for Integrating Models From Other Tools
! Applications in Controls, Signal Processing, Communications, & Other System Engineering Areas
Simulink®Simulink®
The Leading Environment for Modeling, Simulating, The Leading Environment for Modeling, Simulating, & Implementing Dynamic & Embedded Systems& Implementing Dynamic & Embedded Systems
9© 2005 Altera Corporation
Simulink Key FeaturesSimulink Key Features! Hierarchical, Component-Based
Modeling! Open Application Program
Interface (API) ! Hybrid (Mixed-Signal), Multirate
& Multitasking System Simulation! MATLAB Integration! Extensive Library of
Predefined Blocks! Application-Specific
Libraries Available− Video & Image Processing− Signal Processing− Communications− Control − Physical Modeling
10© 2005 Altera Corporation
Design ImplementationRequirements & Specifications
Test & Verification
Model-Based DesignModel-Based Design
Executable Specifications
Automatic Code Generation
Continuous Verification
Design With Simulation
− Reduce Ambiguity Avoid Re-Work
− Rapid Design Iterations
− Minimizes Coding Errors
− Detect Errors Earlier
Continuous VerificationModel Elaboration
11© 2005 Altera Corporation
Streaming Video In/Out
Detection,Thresholding
Tracking, Counting
BackgroundEstimation
Featured Product HighlightVideo & Image Processing BlocksetFeatured Product HighlightVideo & Image Processing Blockset! Provides Over 60 Components & 100�s of Algorithms
Focused on Implementation of Embedded Systems
12© 2005 Altera Corporation
! Analysis & Enhancement! Conversions! Filtering! Geometric Transforms! Morphological Operations! Sinks! Sources! Statistics! Text & Graphics! Transforms! Utilities
Video & Image Processing BlocksetLibraries:
Video & Image Processing BlocksetLibraries:
13© 2005 Altera Corporation
! Estimation! Filtering! Math Functions! Quantizers! Signal Management! Signal Operations! Sinks! Sources! Statistics! Transforms
Signal Processing BlocksetLibraries:Signal Processing BlocksetLibraries:
14© 2005 Altera Corporation
! Optimize Performance & Numerics Due to Finite Word Effects
! Built-In Tools for Scaling & Modeling Finite Word Effects
! Easy to Change Parameters to Simulate Impact of Rounding, Overflow, Etc.
* Signal Processing & Video & Image Processing Blocksets require Simulink Fixed-Point for Integer & Fixed-Point Data Types
Fixed-Point & Integer Modeling*Fixed-Point & Integer Modeling*
15© 2005 Altera Corporation
! Generate C-Code from Video & Image Processing Models− Rapidly Prototype Your Algorithms & Systems− Test & Verify With Target Hardware− Easily Retarget Your Model to Different
Processors
Real-Time Workshop® & RTW Embedded Coder Automatic C-Code GenerationReal-Time Workshop® & RTW Embedded Coder Automatic C-Code Generation
17© 2005 Altera Corporation
Video & Image Processing BlocksetEdge DetectionVideo & Image Processing BlocksetEdge Detection
Live SimulinkDemo
18© 2005 Altera Corporation
Edge DetectionEdge Detection
Model & Simulate Double, Single, Fixed-Point Data Types
Easy to Import Streaming Video Into the Simulation
Handy Viewers for Inspecting Video at Any Point in the Algorithm
Options Important to Embedded System Designers
Executable Executable SpecificationsSpecificationsAccelerating Your Video & Image Processing System Designs
20© 2005 Altera Corporation
Benefits of Executable SpecificationsBenefits of Executable Specifications! Explore Design Trade-Offs in Software Before Costly
Hardware/Prototyping Stage ! Simulate With Different Data Sets (Videos) to Test
Against Multiple Environments, Situations, Etc.! Block Diagram Makes It Easy to Understand Layout of
Algorithm/Design − Reduces Specification Ambiguity− Increases Communication Between Teams/Design Stages
! Facilitates IP-Reuse Between Team Members & Projects
! Copies of return on investment (ROI) white paper available at: www.mathworks.com/roi
22© 2005 Altera Corporation
Video & Image Processing BlocksetDesign & Simulation for Video & Image Processing Systems
Video & Image Processing BlocksetDesign & Simulation for Video & Image Processing Systems! Tools for Model-Based Design:
− Create Executable Specification− Model Floating- & Fixed-Point
Data Types− Trade-Off Analysis & Visualization − Construct Test Harnesses
for Verification− Automatic C-Code Generation for
DSP, µU
ModelModel--Based Design Based Design TutorialTutorialSimulinkSimulink, DSP Builder, DSP Builder& SOPC Builder& SOPC BuilderAlex SoohooDSP Product Marketing ManagerAltera Corporation
24© 2005 Altera Corporation
Altera’s Model-Based DSP Design FlowAltera’s Model-Based DSP Design Flow
DSP SystemDevelopment
SOPC Builder
DSP Builder
25© 2005 Altera Corporation
DSP Builder Design FlowDSP Builder Design Flow
MATLAB/Simulink Domain(System Simulation & Verification)
HDL Domain( Hardware Implementation/ RTL Simulation)
27© 2005 Altera Corporation
Place & RoutePlace & Route
Verify in HardwareVerify in Hardware
Creates SOPC Builder Component
Download Design Hardware-in-the-Loop
Download Design Hardware-in-the-Loop
Creates HDL CodeCreates HDL Code Creates Simulation Testbench
Creates Simulation Testbench
DSP Builder OverviewDSP Builder Overview
HDL SynthesisHDL Synthesis
28© 2005 Altera Corporation
DSP Builder Library ComponentsDSP Builder Library Components! Arithmetic! Bus Manipulation! Complex Signals ! Logical Components! SOPC Ports! Storage! MegaCore® Functions! Rate Change! State Machine! Altera® Library! DSP Board Components
29© 2005 Altera Corporation
SOPC Builder: System Integration ToolSOPC Builder: System Integration Tool
Standard IPFunction
Standard IPFunction
Custom LogicCustom Logic
On-ChipMemoryOn-ChipMemory
MemoryInterfaceMemoryInterface
FlashFlash
SDRAMSDRAM
ControlControl
I/OChannel
I/OChannel
Your Value: Custom Logic
Altera Value: Standard IP Functions & Interconnect Fabric
System on a Programmable Chip
30© 2005 Altera Corporation
Nios II Processor OverviewNios II Processor Overview! Family of 32-Bit
RISC Processors ! Library Peripherals
& Interfaces! Import Custom
Peripherals! Royalty-Free! IDE/Debugger! RTOS! TCP/IP Stack! Performance over
200 DMIPs! Cost As Low as
35¢ of LogicYour Design Here
Nios II
Nios IICPU
On-ChipROM
On-ChipROM
UART
GPIO
Timer
SPI
SDRAMController
Cac
he
Ava
lon
Switc
h Fa
bric
Debug
31© 2005 Altera Corporation
Nios II IDE: Editor & Software Tool-Chain Nios II IDE: Editor & Software Tool-Chain
Integrated Editor & Software Tool-chain Functionality
� Basic Editing Capabilities
� C/C++ Syntax Highlighting
� Comprehensive Search
� Help Feature
Source Editor
� Project Navigation
� Source Files
� Compiler Based on GNU Tool Chain
� Command-Line Operation Possible
Compiler
Features:� Project Manager� Editor & Compiler� Debugger� Flash Programmer
Project Management
32© 2005 Altera Corporation
Building a Real-Time Video SystemBuilding a Real-Time Video System! Target Application:
− Video Edge Detection! Design Steps
− Simulink Testbench− DSP Builder Implementation− SOPC Integration − Control Software Development− Program Hardware Platform
! Hardware Demo
33© 2005 Altera Corporation
DMADMA
31 2RAM
Video Hardware SystemVideo Hardware System
Edge DetectionCo-Processor
Edge DetectionCo-Processor
DMADMA DMADMA
Avalon
DMADMA
SOPC BuilderSOPC Builder DSP BuilderDSP Builder
34© 2005 Altera Corporation
Application SpecificationApplication Specification! Prewitt Edge Detection Algorithm ! VGA Resolution 16-bit RGB(5:6:5) Format Video
− 15 Hz Input Via Camera− 60 Hz Output Via VGA Display Controller
! Sample-Based Algorithm− One Pixel per Clock
! Target Hardware: Stratix® II EP2S60 DSP Development Board
35© 2005 Altera Corporation
Simulink TestbenchSimulink Testbench! Build Model Using Video & Imaging Processing
(VIP) Blockset! Read Video From a File to Matrices Representing
Image (One Matrix per Color Channel)! Outputs Three Displays
− Original Image− Edge Detected Image− Edge Detected Image Overlaid on Original
! Testbench Designed to Match HW Specification− Convert Floating-Point to 5:6:5 RGB− Requires Matrix to Sample Conversion Blocks
37© 2005 Altera Corporation
Testbench – 5:6:5 SamplerTestbench – 5:6:5 Sampler! AVI Reader Outputs RGB Float Matrices for Image
− Specification Requires 5:6:5 Sample-Based Data− Convert Format Using Simulink Blocks
38© 2005 Altera Corporation
Custom Edge Detection Test CoreCustom Edge Detection Test Core! Place Wrapper Around Block From VIP Blockset to
Match HW Specification & Testbench
40© 2005 Altera Corporation
DSP Builder ImplementationDSP Builder Implementation! Substitute Simulink Test Core With DSP Builder Blockset
41© 2005 Altera Corporation
Building the Edge DetectorBuilding the Edge Detector! Consider Vertical Edge Detection
− For Each Pixel in the Image Apply the Following 3x3 Mask on the Intensity I of Adjacent Pixels
10-1
10-1
10-1
42© 2005 Altera Corporation
! Have to Identify 3x3 Pixel Blocks
! Use Delay & Memory Delay Blocks− Delay Uses Registers for Pixel Identification− Memory Delay Uses RAM for Line Storage
Vertical Edge DetectionVertical Edge Detection
Delay
1Z
MemoryDelay
1Z100
Target Pixel(x,y)
I(x-1,y+1)I(x-1,y+1)
I(x-1,y)I(x-1,y)
I(x-1,y-1)I(x-1,y-1)
I(x+1,y+1)I(x+1,y+1)
I(x+1,y)I(x+1,y)
I(x+1,y-1)I(x+1,y-1)
43© 2005 Altera Corporation
A(7:0)
Pipelined Adder
B(7:0)+ R(7:0)
Vertical Edge DetectionVertical Edge Detection! Use DSP Builder Blocks to Carry Out Arithmetic
on Pixel Intensities
− Use Pipelined Adder Blocks
⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜
⎝
⎛
−−+−+
+−
⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜
⎝
⎛
−++++
++
= −
)1,1(
),1(
)1,1(
)1,1(
),1(
)1,1(
),(
yxI
yxI
yxI
yxI
yxI
yxI
yxEdge
45© 2005 Altera Corporation
Vertical Edge Detection BlockVertical Edge Detection Block
Add x+1 Inputs
Add x-1 Inputs
Subtract to Determine Edges
! Horizontal Edge Detection Works Using a Similar 3x3 Mask
47© 2005 Altera Corporation
! DSP Builder Contains Avalon� Port Library− Provides All Signals Required for Avalon Slave
! Other DSP Builder Blocks Used− FIFO Blocks for DMA Performance− Comparison & AND Gates for Control
System Integration – SOPC BuilderSystem Integration – SOPC Builder
Write Data7:0
Datain
Address
Address
d(7:0)
rreq
wreq
q(7:0)full
emptyUsdw(5:0)
FIFO
FIFO
a
b
Comparator
== AND
LogicalBit Operator
48© 2005 Altera Corporation
Creating the Co-ProcessorCreating the Co-Processor! SOPC Slave Block Built Within DSP Builder
− Connected to Tested Edge Detection Unit
49© 2005 Altera Corporation
Building the SOPC SystemBuilding the SOPC System1. Generate SOPC Component Inside DSP Builder
−Using Signal Compiler Block
2. Add Edge Detection Co-Processor Component With SOPC Builder System Editor
3. Add Additional SOPC Components−Nios II Processor−Camera Capture− VGA Device Controller−Memory Controller
50© 2005 Altera Corporation
SOPC Builder System EditorSOPC Builder System Editor
Nios IIEdge
Detector+ DMAs
VGAOut
Camera
51© 2005 Altera Corporation
Nios II Processor Control CodeNios II Processor Control Code! Simple C Language Program
− Rotate Frame Buffers"1 Buffer for Camera to Write To"1 Buffer for Edge Detector Block to Work On"1 Buffer for Display
− Setup DMA Transfers to Processing Module− Push Button Controls
" Toggle On/Off, Video Display
52© 2005 Altera Corporation
Program Hardware: Stratix II DSP Board + Camera Module Program Hardware: Stratix II DSP Board + Camera Module VGA OutputConnector
Peripheral ExpansionDaughter Card Interface
VGA ResolutionCMOS-Sensor
Camera-on-a-Chip Module