视频处理相关技术

Basic Concept Image Sequence（图像序列）

FPS（ Frames Per Second） Resolution 1080p， 720i， DVD， VCD， HD

Compression Codec H.264、MPEG Quick Time、 Real Video、WMV

Real-time Captured / Video files Live stream media

Frames Single Picture/Still Shot, which is shown

as part of a large video or movie Frames are run in succession to produc

e film or video Pixel Color: 8bits, 10bits, 24bits, … FPS

10-16、 24、 30、 70-80、 120

VCD 352x240(NTSC) 352x288 (PAL)

DVD 720x480(NTSC) 720x576(PAL)

HDTV 720p, 1080p, 1080i, 60fps/50fps

Progressive vs Interlaced 逐行扫描 隔行扫描

Video Compression 640x480, 8 bit YUV, 24fps

7.37M bytes/second 26.5G bytes/hour

MPEG-1 187K bytes/second

720p HDTV, 60fps, progressive 1.3Gbits/second HDTV: 20Mb/s bandwidth

Compression by a factor of 70

Compression Lossless/Lossy

5 4 2 -> compression -> decompression -> 3 2 0, high error 4.9 4.1 1.9, low error 5 4 2, lossless

Codecs H.261, H.263, H.264（视频编码专家组） MPEG-1,2,..,4（活动图像编码专家组）

DivX, XviD,…

Real Video QuickTime

Special Requirement Real-time compression -> Real-time

decompression Video Meeting Chatting

Real-time decompression with High compression rate/ High Quality Movie Film

Fields Theory Decoder/Encoder Development DirectShow / Mplayer / FFMpeg, etc Application using OpenCV

DirectShow A Simple Movie Player A Simple Camera Player

DirectShow Part of DirectX before

Direct3D, DirectShow, DirectInput, DirectSound, …

Removed to Windows Platform SDK in 2005 COM based Interface based

Framework

DirectShow Filters The basic building block, which can

Read files Get video from a video capture device Code/decode streams Pass data to another filter Pass data to the graphics or sound card Input/Output Pins

Filter Graph

Program // Create the filter graph hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER,

IID_IGraphBuilder, (void **) & g_pGraph); // Create the capture graph builder hr = CoCreateInstance(CLSID_CaptureGraphBuilder2 , NULL,

CLSCTX_INPROC_SERVER, IID_ICaptureGraphBuilder2, (void **) & g_pCapture);

hr = g_pGraph->AddFilter(pSrcFilter, L"Video Capture"); hr = g_pCapture->RenderStream(&PIN_CATEGORY_PREVIEW,

&MEDIATYPE_Video, pSrcFilter, pGrabber, pNullRender);

Tools GraphEditPlus

Generate Source Code in C++/C# on GraphEdit

DirectShow.net C# development

OpenCV Video Files

FFMpeg based

Cameras DirectShow based

Input Source VideoCapture video(0); VideoCapture video(1); VideoCapture video(“hello.avi”);

Read Frame cv::Mat frame; video.read(frame);

Process Frame Just Like Image Sample

Face Detector