time to impact - ece631web.groups.et.byu.netece631web.groups.et.byu.net/lectures/ecen631 26 - time...
TRANSCRIPT
€
z'= X fx'
=zxf⋅fax
=za
a =zz'
=x'x
=y'y
Time to Impact
3/7/11 ECEn631 3
Ignoringtheloca.onofthefocalcenter, x and y can be the width and height of the object.
The time to impact, τ, is the distance, z, divided by the velocity, v:
Time to Impact
3/7/11 ECEn631 4
If all we know is the frame rate, then this time to impact is measured as number of frames.
The expansion rate is not constant over time: for an object approaching the camera at a fixed velocity, v, a(t) is given by
, where z0 is the value of z at t = 0.
Time to Impact
3/7/11 ECEn631 5
The time to impact, τ, is the distance, z, divided by the velocity, v:
If we know the speed (or distance traveled between frames) that the camera is approaching the object (from onboard GPS or encoder), we can also estimate the distance to the object from the expansion rate.
Time to Impact
3/7/11 ECEn631 6
€
distance = τ × v = τ × (z − z' )
=aa−1
× (z'−z) =z
z'−z× (z'−z) = z
If the camera is calibrated and the object size is known, we can estimate the distance to the object from the object image size.
Time to Impact
3/7/11 ECEn631 7
€
z (mm) = f (pixel) X(mm)x (pixel)
Feature Identification
Calculateimagederiva.ves,IxandIyusing3x3Sobelkernel(inhardware)
Iden.fyfeaturesusingHarris’cornerdetectorforeachpixel:
3/7/11 ECEn631 8
Feature Tracking Foreachframe,trackthehighestHarrisresponsewithinnpixelsofthepreviousfeature
3/7/11 ECEn631 10
3D Reconstruction Foreachfeature,trackthedistanceofthefeaturefromtheop.calcenterforeachframeasvehiclemoves(centercanbeobtainedfromcameracalibra.onorusetheimagecenter)
3/7/11 ECEn631 11
3D Reconstruction Foreachfeature,trackthedistanceofthefeaturefromtheop.calcenterforeachframeasvehiclemoves
3/7/11 ECEn631 12
3D Reconstruction Foreachfeature,trackthedistanceofthefeaturefromtheop.calcenterforeachframeasvehiclemoves
3/7/11 ECEn631 13
3D Reconstruction Foreachfeature,trackthedistanceofthefeaturefromtheop.calcenterforeachframeasvehiclemoves
3/7/11 ECEn631 14
3D Reconstruction
Collectthedistancesfromtheop.calcenterasrobotmoves
3/7/11 ECEn631 15
Dis
tanc
e Fr
om O
ptic
al C
ente
r
Robot Distance Traveled (from wheel encoders)
3D Reconstruction
Collectthedistancesfromtheop.calcenterasrobotmoves
3/7/11 ECEn631 16
Dis
tanc
e Fr
om O
ptic
al C
ente
r
Robot Distance Traveled (from wheel encoders)
3D Reconstruction
Uselinearleast‐squarestogetclosed‐formapproxima.ontover.calasymptote
3/7/11 ECEn631 17
Dis
tanc
e Fr
om O
ptic
al C
ente
r
Robot Distance Traveled (from wheel encoders)
Distance to point
3D Reconstruction
Giventhedistance(Z),intrinsicparameters(focallengthandop.calcenter),andtheimagepoints(x,y)wecandeterminethe3Dposi.onofthepoint(X,Y,Z)
3/7/11 ECEn631 18
Hardware Implementation
Howdoyoucalculatereal‐.me.me‐to‐impactonasmallautonomousvehicle?– Real‐.me:15‐30fpsorhigherlivevideofeed
GeneralPurposeCPUorGPU?– Toolarge,powerhungry,ornotfastenough
CustomSiliconVisionProcessor– Tooexpensivetodeveloporproduceinsmallquan..es
3/7/11 ECEn631 23
Solution FPGA‐basedRobo.cVisionPlaTorm
– OneormoreembeddedCPUs– Customhardwareaccelera.on– CompleteSystemonaProgrammableChip(SOPC)
3/7/11 ECEn631 24
Robot Platform EmbeddedFPGACores
– XilinxMicroBlazeProcessor@75Mhz– CustomFloa.ngPointUnit(FPU)– 64KBRAMMemory– DDRSDRAMController– MotorController– CameraController– SerialInterfaces(UART,SPI)– TimersandInterruptControllers
3/7/11 ECEn631 27
Image Derivatives
3/7/11 ECEn631 31
DMA data
Dual Port BRAM Buffer
External Camera
RAM Write
RAM Read
pixel
Ix Pixel Matrix
Iy
FIFO
FIFO
FIFO
DMA Control
DMA address FPG
A
Image Derivatives
3/7/11 ECEn631 32
DMA data
Dual Port BRAM Buffer
External Camera
RAM Write
RAM Read
pixel
Ix Pixel Matrix
Iy
FIFO
FIFO
FIFO
DMA Control
DMA address FPG
A
Image Derivatives
3/7/11 ECEn631 33
DMA data
Dual Port BRAM Buffer
External Camera
RAM Write
RAM Read
pixel
Ix Pixel Matrix
Iy
FIFO
FIFO
FIFO
DMA Control
DMA address FPG
A
Image Derivatives
3/7/11 ECEn631 34
DMA data
Dual Port BRAM Buffer
External Camera
RAM Write
RAM Read
pixel
Ix Pixel Matrix
Iy
FIFO
FIFO
FIFO
DMA Control
DMA address FPG
A
Image Derivatives
3/7/11 ECEn631 35
DMA data
Dual Port BRAM Buffer
External Camera
RAM Write
RAM Read
pixel
Ix Pixel Matrix
Iy
FIFO
FIFO
FIFO
DMA Control
DMA address FPG
A
Resource Usage and Performance of Image
Processor XilinxVirtex‐II(XC2V1000‐6FG456)
– 234Slices(4.6%)– 231LUTs(2.3%)– 1BRAM(2.5%)– MaximumFrequency:170Mhz– 75MHzsufficientfor320x240,30fps
PixelProcessoronly,excludingDMAControlleranditsFIFOs
3/7/11 ECEn631 36
Reconfigurable Platform Systemcanbereconfiguredtosupportalternatealgorithms– Otherkernels(e.g.,7x7)– Blockmatching– Stereovision– Acombina.onofalgorithmsinparallel
3/7/11 ECEn631 37