soccer trajectory
Post on 25-Jan-2015
782 Views
Preview:
DESCRIPTION
TRANSCRIPT
Trajectory Analysis of Broadcast Soccer Videos
Computer Science and Engineering DepartmentComputer Science and Engineering Department
Indian Institute of Technology, KharagpurIndian Institute of Technology, Kharagpur
by
Prof. Jayanta Mukherjee
jay@cse.iitkgp.ernet.in
Collaborators
•V. Pallavi --- research scholar. •Prof. A.K. Majumdar, CSE •Prof. Shamik Sural, SIT
OUTLINE• Motivation and Objective
• State Based Video Model
• Extraction of Features
• Trajectory Detection
• States and Event Detection
MotivationIncreasing availability of soccer videos
Soccer videos appeal to a large audience
Processing of soccer videos to deliver it over narrow band networks
Relevance of soccer videos drops significantly after a short period of time
Therefore soccer video analysis needs to be made automatic and the results must be semantically meaningful
State based Video Model
Video data model : representation of information contained in the unstructured video in order to support users’ queries.
State based model: states of soccer video objects and their transitions (due to some event).
State Chart Diagram for Ball Possession
Immediate Goal
Our objective is to identify these states and their transitions by analyzing the unstructured video.
Cinematic Features
– Shot Transitions
– Shot Types
– Shot Durations
Object Based Features
– Players
– Ball
– Billboards
– Field Descriptors
Features UsedFeatures Used
Shots can be classified into:
• Long shot
– Captures a global view of the field
• Medium shot
– Shows close up view of one or more players in a specific part of the field
• Close shot
– Shows an above-waist view of a single player
Shot classification
Typical long views in soccer videos
Grass covering entire frame Grass covering partial frame
Shot Classification (contd..)Shot Classification (contd..)
Soccer Video Sequence
If dominant color is green
Dominant color ratio >0.75 and
<=1.0
Long Shot Medium Shot
Dominant color ratio >0.5 and
<=0.75
Close Shot
Dominant color ratio >0.25 and
<=0.5
Shot Classification Results
20147211078Close Shot
73841837Medium Shot
14424325830Long Shot
Unclassified Shot
(No of frames)
Close Shot
(No of frames)
Medium Shot
(No of frames)
Long Shot
(No of frames)
Predicted ClassTrue Class
87.6387.63Close ShotClose Shot
83.7683.76Medium Medium ShotShot
96.6896.68Long ShotLong Shot
% of True % of True ClassificationClassification
Shot TypeShot Type
Shot Classification Results
Shot detection improved by shot classification
Object Based FeaturesFeature extraction for grass pixels
Each frame is processed in YIQ color space.
It is found experimentally that grass pixels have ‘I’ values ranging between 25 and 55 while ‘Q’ values range
between 0 and 12.
0 50 100 150 200 2500
50
100
150
200
250
Q v
alu
es
I values
Grass Values
Playfield region detected
Grass pixels detected for a long view frame
Object Based Features (contd..)
Playfield Line Detection
A playfield line separates playfield from the non playfield background which are usually the billboards (also called advertisement boards).
Hough transform is used to detect the playfield line.
Object Based Features (contd..)
Midfield line is the line that divides the playfield in half along its width. Hough transform is applied to detect the midfield line.
Midfield Line Detection
Ball DetectionObject Based Features (contd..)
Challenges :
Features of the ball (color, size, shape) vary with time
Relative size of the ball is very small
Ball may not be an ideal circle because of fast motion and illumination conditions
Objects in the field or in the crowd may look similar to a ball
Field appearance changes from place to place and time to time
No definite property to uniquely identify ball in a frame
Detecting Ball Candidates in Long Shots
• Obtain ball candidates by detecting circular regions by using circular Hough Transform
• Filter the non ball candidates by :
– Removing candidates from channel’s logo
– Removing candidates from gallery region
– Removing candidates from midfield line
– Filtering out the candidates moving against the camera
Object Based Features (contd..)
Ball candidates before and after filtering
Ball candidates before filtering Ball candidates after filtering
Object Based Features (contd..)
Detecting Players in Long Shots
Challenges
• Features of the players (color, texture, size, motion) are neither static nor uniform
• Players appear very small in size
• Size of players changes with their position and zooming of cameras
• Color and texture of the jersey and shorts vary from team to team
• Players in the field do not have constant motion
Object Based Features (contd..)
• Obtain player pixels by removing non player pixels :
– Removing grass pixels
– Removing the broadcasting channel’s logo
– Removing the extra field region (billboards and gallery)
– Removing pixels from the midfield line
• Segment the image containing player pixels to isolated player regions by :
– Region growing algorithm
– Center of the bounding rectangle of each region is said to be the location of the player
Detecting Player Regions
Object Based Features (contd..)
A Long Shot View
Object Based Features (contd..)
Player pixels detected
Object Based Features (contd..)
Players detected in long shot views
Object Based Features (contd..)
Team Identification in Soccer Videos
Players in a soccer videos are classified using a supervisory classification method.
Mean I and Q values of the player regions are obtained by randomly selecting a few frames
The minimum and maximum I and Q values are set as the range for classifying player regions
Feature Detection (Contd.)
Team Classification in Soccer Videos
Experiments were performed on two different matches:
Real Madrid and Manchester United (UEFA Champions League 2003)
Chelsea and Liverpool (UEFA Champions League 2007)
Feature detection (contd.)
Team Classification Results
72
(11.51)
6520Team B
50
(25)
3
(17.86)
Unclassified
173
(16.29)
0725Team A
UnclassifiedNo of players (%)
Team BNo of players (%)
Team ANo of players (%)
Predicted ClassTrue Class
Real Madrid and Manchester United
Chelsea and Liverpool
72
(9.94)
6520Team B
503
(37.5)
Unclassified
173
(19.27)
0725Team A
UnclassifiedNo of players (%)
Team BNo of players (%)
Team ANo of players (%)
Predicted ClassTrue Class
Team Classification Results (contd..)
Camera Related FeatureObject Based Features (contd..)
Camera Direction Estimation :
1. Optical Flow velocities and their directions are computed using Horn and Shunck’s method.
2. Based on the sign of the horizontal component of the majority pixels in a frame, the direction of movement (left or right) of the camera is estimated.
Camera Direction Estimation (contd..)
Optical flow velocities for the camera moving towards right
Tracking of Broadcast Video Objects
Challenges
Camera parameters are unknown
Cameras are not fixed
Cameras are zoomed and rotated
Broadcast video is an edited video
Construction of a Directed Weighted Graph
Objects in a frame form nodes.
Between two correlated objects in two different framesan arc (edge) is formed.
The measure of correlation or similarity provide the weight.
Temporal direction provides the direction of the edge.
Object Trajectory Detection
Given a source node, longest path of the graph obtained by dynamic programming gives the path of the object.
Tracking of Broadcast Video Objects (contd..)
Ball detection results for long shots
10010027272741400-41940
91.8598.03609597650Liang et al * sequence 1
95.8398.26689677719Liang et al * sequence 2
94.7494.7419181937020-37400
10010020202040500-40900
92.7396.2353515535900-37000
93.3393.3330283034800-35400
91.395.4522212330300-30760
9094.7319182023800-24200
PrecisionRecallBall present in
(number of frames)
Ball identified in
(number of frames)
Total
(number of frames)
Frame Range
Average Recall is 96.75 % and Average Precision is 94.42 % * Liang D., Liu Y., Huang Q. \& Gao W., A Scheme for Ball Detection and Tracking in Broadcast Soccer Video, Pacific Rim Conference on Multimedia, 2005, 1, LNCS 3767, 864-875.
Tracking of Broadcast Video Objects (contd..)
Results for ball detection in long shots (contd..)
Given a source node (player in the first frame), longest path of the graph obtained by dynamic programming gives the path of the player in the whole sequence.
Tracking a Single Player
Player being tracked
Tracking of Broadcast Video Objects (contd..)
Tracking Multiple Players
Longest path from each node (represented by players in the first frame) of the graph obtained by dynamic programming gives the trajectories of the players for the sequence of frames.
Limitations :
Occlusion between players
Players in contact
Similarity between players belonging to same team
Tracking of Broadcast Video Objects (contd..)
Resolving Conflicting Player Trajectories
If more than one player has more than two common nodes in its trajectory then only one amongst them is true.
The path having maximum weight is said to be the true trajectory
Nodes constituting the paths of correctly detected players are removed and a graph is again constructed
Mistracked players are again tracked
Tracking Multiple Players (contd..)
Multiple Player Detection Results
99.3799.48158815781586317Soccer 6
95.2810053050550568Soccer 5
83.58100670560560100Soccer 4
10085.7136036042060Soccer 3
81.81100220018001800200Soccer 2
81.03100311025202520180Soccer 1
Precision (%)
Recall (%)
SOP detected SOTP detectedSOP presentNo of FramesVideo file
Average Recall is 97.53 % and Average Precision is 90.18%
Tracking Multiple Players (contd..)
Multi - Player Tracking Results
96.59153212901586317Soccer 6
91.4946235650568Soccer 5
100560540560100Soccer 4
90.4838030042060Soccer 3
100180012001800200Soccer 2
85.71216018002520180Soccer 1
Accuracy (%)
SOTP tracked by tracking
and retracking algorithm
SOTP tracked by tracking algorithm
SOP presentNo of FramesVideo file
Average Accuracy is 94.05 % .
Tracking Multiple Players (contd..)
Occlusion Results
1002020Soccer 6
801620Soccer 5
1004444Soccer 3
55.552036Soccer 1
Accuracy
(%)
No of cases that could be solved
No of occlusion and contact cases
Video file
Average Accuracy is 83.89 % .
Tracking Multiple Players (contd..)
Multi - Player Tracking Results
Multi - Player Tracking with Occlusion Results
Multi - Player Tracking with Occlusion Results (contd..)
Multi - Player Tracking with Occlusion Results (contd..)
Multi - Player Tracking with Occlusion Results (contd..)
Tracking the Mistracked Player (contd..)
Tracking the Mistracked Player (contd..)
Tracking the Mistracked Player (contd..)
Tracking the Mistracked Player (contd..)
Detection of States and Events
The features extracted and the trajectories detected are used to detect states and events based on the proposed state based video model.
States identified
- Ball possession states
Events detected
- Ball passing events
State Chart Diagram for Ball Possession
Detection of States and Events (contd..)
State Detection
Ball possession states are obtained based on
Spatial proximity analysis:
Distance between nearest player and second nearest player to the ball
Spatial arrangements between the players and the ball
Ball Possession State Detection
Ball in possession of player 1’s team
Ball in possession of player 1’s team
Ball Possession State Detection
Ball in possession of player 1’s team
Ball in a fight state
Ball Possession Results
8
(1.92)
39216
(3.85)
Team B
5412
(17.14)
4
(5.71)
Fight
8
(3.17)
2
(0.93)
206Team A
FightNo of frames
(% of misclassified
frames)
Team BNo of frames
(% of misclassified
frames)
Team ANo of frames
(% of misclassified
frames)
Predicted ClassTrue Class
Edit Distance as performance measure for ball possession states
If the actual state sequence for a sequence of frames is:
AAAAFFFFFFFFFFBBBB
And if the state sequence obtained by the proposed algorithm is:
AAAAFFFFFFFFBBBBBB
Both the sequences are represented as strings S1 and S2.
Edit distance D(S1, S2 ) is defined as the minimum number of point mutations required to change S1 to S2 where a point mutation is one of:
replacing an alphabet
inserting an alphabet
deleting an alphabet
Edit distance for the above sequence is 2. While normalized edit distance is:
D(S1, S2 )/| S1 |
|S|)S,S(D 121
Shot wise ball possession results
Event Detection
The event detected in this work is the ball passing event. It can be:
Forward pass
Reverse pass
Event Detection (contd..)
The ball passing event cannot be detected
from state transition graphs because: Ball is usually passed between players of the same team State transition graphs show the change in ball possession states from Team A-Team B, Team B - Team A, Team B – Fight , Fight – Team B, Team A – Fight or Fight – Team A
Schematic diagram for ball passing events
Ball is said to be passed in a sequence of frames, if:
Nearest player in the initial frames of the sequence is the second nearest player to the ball in the subsequent frames
Nearest and the second nearest players to the ball belong to the same team
Example of a ball passing event:
Example of a ball passing event (contd..)
Example of a ball passing event (contd..)
Example of a ball passing event (contd..)
Classifying ball passing events
Forward pass:
Direction of camera motion is towards the goal post of the team opposite to that of the nearest player
Reverse pass:
Direction of camera motion is towards the goal post of the team of the nearest player
Results for ball passing events
Average Recall = 100% and Precision = 60%
Classification of ball passing events:
5-Reverse
13Forward
Reverse
(no of passes)
Forward
(no of passes)
False Ball PassesTrue Ball Passes
Graphs for ball possession and ball passing
Graphs illustrating ball possession states and ball passing events for Sequence 7
Graphs for ball possession and ball passing
Graphs illustrating ball possession states and ball passing events for Sequence 10
top related