a robust scene-change detection method for video segmentation chung-lin huang and bing-yao liao ieee...
Post on 20-Dec-2015
215 views
TRANSCRIPT
A Robust Scene-A Robust Scene-Change Detection Change Detection Method for Video Method for Video
SegmentationSegmentation
Chung-Lin Huang and Bing-Yao Chung-Lin Huang and Bing-Yao LiaoLiao
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
OutlineOutline
• IntroductionIntroduction• Abrupt Scene-Change DetectionAbrupt Scene-Change Detection• Gradual Scene-Change DetectionGradual Scene-Change Detection• Experimental ResultsExperimental Results• ConclusionConclusion
IntroductionIntroduction
• The main problem of segmenting a The main problem of segmenting a video sequence into shots is the video sequence into shots is the ability to distinguish between ability to distinguish between scene scene breaks and normal changesbreaks and normal changes that that happen in the scenehappen in the scene
• This paper combines the This paper combines the intensityintensity and and motion informationmotion information to detect to detect scene changes such as abrupt scene scene changes such as abrupt scene changes and gradual scene changeschanges and gradual scene changes
Previous ProblemsPrevious Problems
• The two main problems in most existing The two main problems in most existing algorithmsalgorithms– they are they are threshold-dependent algorithmsthreshold-dependent algorithms– they they suffer false detectionsuffer false detection with scenes involving with scenes involving
fast camera or object motion.fast camera or object motion.
• This paper proposes a scene-change This paper proposes a scene-change detection algorithm with three contributionsdetection algorithm with three contributions– Relaxing thresholdRelaxing threshold selection problem selection problem– higher detection rate (scene change should not be higher detection rate (scene change should not be
missed)missed)– lower false alarm ratelower false alarm rate
ABRUPT SCENE-CHANGE ABRUPT SCENE-CHANGE DETECTIONDETECTION
• MethodMethod– Measurement of the Changes Between Measurement of the Changes Between
FramesFrames• Pixel-Based DifferencePixel-Based Difference• Histogram-Based DifferenceHistogram-Based Difference
– Static Scene TestStatic Scene Test– Scene Transition ClassificationScene Transition Classification– Detection AlgorithmDetection Algorithm
Detection AlgorithmDetection Algorithm• First phaseFirst phase
– locate the highest and the second highest peaks of DClocate the highest and the second highest peaks of DCimage difference in the midst of the sliding window, image difference in the midst of the sliding window, and then and then calculate the ratio ncalculate the ratio n between the first and se between the first and second peakscond peaks
• Second phaseSecond phase– Histogram MeasureHistogram Measure– Static Scene Test Static Scene Test (Most of the false alarms declared by the hi(Most of the false alarms declared by the hi
stogram detector are due to sudden light changes, while the edgstogram detector are due to sudden light changes, while the edge information is more or less invariant to these changes)e information is more or less invariant to these changes)
– Scene TransitionScene Transition
genuine AmbiguousNo Scene Change
Nhigh Nlow
Measurement of the Measurement of the Changes Between FramesChanges Between Frames
• Pixel-Based Difference:Pixel-Based Difference:–
• Histogram-Based Difference: (XHistogram-Based Difference: (X2 2 Test)Test)–
Where Cx and Cy are the DC image of frames X and Y
Color HistogramColor Histogram
• Efficient representationEfficient representation– Easy computationEasy computation– Global color distributionGlobal color distribution
• Insensitive toInsensitive to– RotationRotation– ZoomingZooming– Changing resolutionChanging resolution– Partial occlusionsPartial occlusions
• DisadvantageDisadvantage– Ignore spatial relationshipIgnore spatial relationship– Sensitive in illumination changesSensitive in illumination changes
• Choose illumination-insensitive color channelsChoose illumination-insensitive color channels
ExampleExample
• Color space selection & quantizationColor space selection & quantization– Use RGB channelsUse RGB channels
• Each channel is divided into 2 intervalsEach channel is divided into 2 intervals• Total number of bins = 2Total number of bins = 233 = 8 = 8
– HH((II): Color histogram for Image ): Color histogram for Image II
– HH11 = (7, 7, 7, 7, 9, 9, 9, 9) = (7, 7, 7, 7, 9, 9, 9, 9)
• Image 1 has 7 pixels in each Image 1 has 7 pixels in each CC11 to to CC44, and 9 , and 9 pixels in each pixels in each CC55 to to CC88
Static Scene TestStatic Scene Test
• Define Define – all objects present in the scene exhibit rather all objects present in the scene exhibit rather
small motion compared to the frame size, and small motion compared to the frame size, and global movement caused by the camera is slow global movement caused by the camera is slow and smooth.and smooth.
• MethodMethod– Edge DetectionEdge Detection– Edge DilationEdge Dilation
• ResultResult– The transition of two consecutive frames with The transition of two consecutive frames with
covering ratiocovering ratio larger than a larger than a predefined thresholdpredefined threshold is considered as a non-static or dynamic scene.is considered as a non-static or dynamic scene.
Example of edge Example of edge detectiondetection
Edge Detection
Edge Dilation ( r=3 )
Scene Transition Scene Transition ClassificationClassification
• Transition TypeTransition Type– 1) static scene to static scene1) static scene to static scene– 2) dynamic scene to static scene or vice 2) dynamic scene to static scene or vice
versaversa– 3) 3) dynamic scene to dynamic scenedynamic scene to dynamic scene
• Dynamic-to-dynamic transition usually Dynamic-to-dynamic transition usually indicates continuous object or camera indicates continuous object or camera motion, rather than a real scene changemotion, rather than a real scene change
Gradual Scene-Change Gradual Scene-Change DetectionDetection
• Gradual Scene-ChangeGradual Scene-Change– DissolveDissolve– Fade-InFade-In ( X = 0 )( X = 0 )– Fade-OutFade-Out ( Y = 0 )( Y = 0 )
• Why not easy to detectWhy not easy to detect– Camera and object motions always introduce Camera and object motions always introduce a a
larger variationlarger variation than a gradual transition. than a gradual transition.
Scene X to Y In Duration T
Intensity Statistics Intensity Statistics ModelModel
• Normal CaseNormal Case– For any frames near the reference frame, For any frames near the reference frame,
their their dissimilarity measure almost dissimilarity measure almost increases exponentiallyincreases exponentially with their with their distancedistance
• Gradual TransitionGradual Transition– The dissimilarity The dissimilarity increases linearlyincreases linearly with with
their distance during the transitiontheir distance during the transition– After the transition is over, the difference After the transition is over, the difference
measures are measures are randomly distributedrandomly distributed
Normal Sequence
Gradual Transition
Seed : the beginning frame of a gradual
transition
• N-distance measureN-distance measure– the difference measure generated by comparing a the difference measure generated by comparing a
frame with itself and its successive ( N – 1 ) framesframe with itself and its successive ( N – 1 ) frames– – Ideal model of the N-distance measureIdeal model of the N-distance measure
Gradual Scene-Change-Gradual Scene-Change-Detection AlgorithmDetection Algorithm
• 1) 1) N-distance measureN-distance measure• 2) Difference operation2) Difference operation
• 3) 3) Low-pass filtering: Implement a simple low-pass filter to keep the low frequency segments and to remove the high-frequency segments
Gradual Scene-Change-Gradual Scene-Change-Detection AlgorithmDetection Algorithm
• If the number of zero crossings between frame k and frIf the number of zero crossings between frame k and frame l (k and l is zero crossing frame)ame l (k and l is zero crossing frame)– Zero-crossing rate calculation:Zero-crossing rate calculation:
– larger than a threshold : larger than a threshold : high frequencyhigh frequency fragment fragment– else : else : low frequencylow frequency fragment (gradual scene-change segment) fragment (gradual scene-change segment)
• a local “score” record mechanisma local “score” record mechanism– ScoreScoreii(q)(q) q=0, 1, 2 …, N-1q=0, 1, 2 …, N-1
• High frequency fragment : ScoreHigh frequency fragment : Scoreii(q) = 0(q) = 0• Low frequency fragment : ScoreLow frequency fragment : Scoreii(q) = 1(q) = 1
0
50
100
150
1 3 5 7 9 11 13 15 17 19 21 23 25
-10
0
10
20
30
40
1 3 5 7 9 111315171921 232527
0
0.5
1
1.5
1 3 5 7 9 11131517192123 2527
N-Distance Measure
Difference operation
Local Score
Gradual Scene-Change-Gradual Scene-Change-Detection AlgorithmDetection Algorithm
• a global “track” record mechanisma global “track” record mechanism– Track(p)Track(p) p = 1, 2, 3, …,Lp = 1, 2, 3, …,L
• After every N-distance Measure of frameAfter every N-distance Measure of frame i i ,, we can get the local Score we can get the local Scoreii(q)(q)– Accumulate the Score record to the Track recordAccumulate the Score record to the Track record
the total number of frames in the video sequence
Improve Gradual Scene-Improve Gradual Scene-Change-Detection Change-Detection
AlgorithmAlgorithm• To develop a fast seed-searching processTo develop a fast seed-searching process
– we select one from every S consecutive frames for N-distance measure.
• Since gradual scene change does occur in Since gradual scene change does occur in segment 3 only, we need to ignore the segment 3 only, we need to ignore the scores in segment 1 due to correlation scores in segment 1 due to correlation behavior of the reference frame and its behavior of the reference frame and its neighboring frames.neighboring frames.
• The correlated distance in segment 1 is C
EXPERIMENTAL EXPERIMENTAL RESULTS (1)RESULTS (1)
EXPERIMENTAL EXPERIMENTAL RESULTS (2)RESULTS (2)
Performance Measure
Performance Result
EXPERIMENTAL EXPERIMENTAL RESULTS (3)RESULTS (3)
ConclusionConclusion
• This method avoided the false alarms This method avoided the false alarms by using the by using the validation mechanismvalidation mechanism..
• It also proves that the It also proves that the statistical statistical model-based approachmodel-based approach is reliable for is reliable for gradual scene-change detection.gradual scene-change detection.
• Experimental results show that a Experimental results show that a very very high detection ratehigh detection rate is achieved while is achieved while the false alarm rate is comparatively the false alarm rate is comparatively low.low.