smart camera as embedded system
DESCRIPTION
Smart cameras capture high-level descriptions of a scene and perform real-time analysis of what they see. These low-cost, low-power systems push the design space in many dimensions, making them a leading-edge application for embedded system research.TRANSCRIPT
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Smart Cameras as Embedded Systems
B. SwarnalathaM.Tech VLSI12651D5710
Adams Engineering CollegePaloncha.
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Image
Video
Digital Camera & Smart Camera
Detection and recognition algorithms
Low level processing
High level processing
Towards and Embedded System
Requirements
Components
Core Processor TriMedia
Advancements in VLSI required to implement
Embedded Smart camera
Overview of Seminar
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
IMAGE
10 12 35 54 34 23 201 2 10 124 5 6 7 8 9 9 9 0 087 6 8 0 7 68 8 9 09 65 87 88 7 9 9 8 8 8 88 8 8 8 8 89 9 90 0 000 5 54 4 55 6 76 7 43 65 7 7 89 7 6 6 8 997 6 6 6 78 9 166 6 77 64 44 4 5 55 43 2 54 87 545 6 54 67 45 7 3 7 98 54
An image may be defined as a two-dimensional function f(x, y) where x and y are spatial (plane) coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image at that point. A small region in the image is shown in matrix.
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
VIDEO
Video signal is a representation of a scene with time as a third parameter.
V = (x, y, t)
• The parameters of a video signal are:
• Frame size: 560x480 & frame rate: 30 fps
• Video signal format:Analog & Digital
Video capture devices are Sony Canon samsung
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Digital Camera and Smart Camera
Digital Cameras capture only images as digital files that users can upload to their Computer , manipulate with software and distribute electronically. Eg. Nikon
Smart Cameras capture high-level descriptions of the scene and analyze
What they see.
These devices could support a wide variety of applications including
human and animal detection,
surveillance,
motion analysis, and
facial identification
Eg. Windows for KINECT Digital CameraSMART Digital Camera
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Detection and Recognition Algorithms
Low Level Processing
• Region Extraction
• Contour Following
• Ellipse fitting
• Graph Matching
High Level Processing
• Hidden Markov Models
• Classifiers
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
DETECTION AND RECOGNITION
ALGORITHMS
Low level processingRegion Extraction
The subject to be identified for gesture recognition in the video frames are extracted. Fig2. (b)•Contour following
Grouping of pixels into contours that geometrically define the regions. Fig2. (c)•Ellipse fitting
To correct for deformations in image processing caused by clothing, objects in the frame, or some body parts blocking others, an algorithm fits ellipses to the pixel regions to provide simplified part attributes. Fig2. (d)•Graph matching
Meaningful feature vectors are extracted from the modeled body parts with ellipses. Figure2
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
High level Processing
The high level processing component , which can be adopted to different
Applications, compares the motion pattern of each body part.
Example, a pointing gesture could be recognized as a command
to “go to the next slide” in a smart meeting room
or “open the window” in a smart car ,
Where as a smart security camera might interpret the
gesture as suspicious or threatening.
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Human Detection and Activity Recognition Algorithm
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Towards an Embedded Smart Camera
RequirementsFrame rate: The embedded smart camera system must process a certain amount of frames per second to properly analyze motion and provide useful resultsLatency: The amount of time takes to produce a result from the processed frames
Components•100 MHz Philips TriMedia TM-1000
as video processor.•Hi8 Cameras•Shared memory interface•Host computer•Debugging algorithms and
programs•Networked system for connecting
multiple cameras
Smart Meeting Room
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
TriMediaProcessor TM-1000
The Philips TriMedia TM1000 family of devices have a higher performance Very Long Instruction word (VLIW ) core.
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
TriMediaProcessor TM-1000
•a 5-issue VLIW architecture with a 32-bit word size;
•27 functional units, offering a choice of operation types
in each slot in the instruction;
•any operation can be guarded to provide conditional
execution without branching;
•instruction set and functional units optimized with
respect to media processing;
•a single multi-ported register file with bypass network,
allowing 1-cycle latency operations;
•32 kB, 8-way instruction cache;
•16kB, 8-way, quasi-dual ported, data cache;
•a variable-length (compressed) instruction set design
Features
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Advancements in VLSI required to implement a smart camera embedded system
Algorithmic challengesChoosing best computationally efficient algorithm with minimum memory usage.
Library FunctionsThe library function of the processor should provide special function to provide Instruction level-parallelism
Eg1. INONZERO: This instruction takes two input operands. If the first is non zero, the destination is set to the value of the second operand; otherwise it is set to zero.
Eg2. IABS: This instruction can provide absolute values
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Advancements in VLSI required to implement a smart camera embedded system
Control-to-data transformationThe data transformation can be controlled using a processor with more functional units.
Designing SINGLE-INSTRUCTION MULTIPLE-DATA (SIMD) Processors to achieve real time performance.
Existing processors like (80xx, INTEL ) are pixel plane processors. These processor can perform arithmetic and logical operations with limited no of operands.
Embedding single-instruction multiple-data (SIMD) processors into sensors is critical to improve the real time performance.
Processor architectures are to be designed to take multiple data in one instruction to increase the computation power required to meet the real time challenges.
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
Reference: Wayne Wolf, Burak Ozer, Tiehan Lv “Smart Camera as EmbeddedSystems”. IEEE Trans. Vol. 45, 2002.
Copy right IEEE
ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710
THANK YOU