1 video quality for public safety applications margaret pinson public safety communications research...
TRANSCRIPT
1
Video Quality for Public Safety Applications
Margaret Pinson
Public Safety Communications Research
U.S. Dept. of Commerce
June 6, 2011
2
Public Safety Communications Research Program
VISION
Public safety pracitioners• Police• Firefighters• Emergency Medical Services
Seamless exchange of voice, video and data to effectively respond to any incident or emergency
By encouraging the development and adoption of critical standards
3
PSCR PortfolioPSCR Portfolio
LMR Standards and Technologies
Broadband Standards and Technologies
Interoperability Device Standards and Technologies
Emerging Standards and Technologies
Cross-cutting or Supporting Activities
P25 CAP 700 MHz Broadband Multi-Band Radio P25 Security
Program Management &
ReportingProject 25 (P25)
Standards Development
Public Safety VoIPInterim
Interoperability Device Testing
Technical Services Projects
Statement of Requirements
(SOR)
ISSI Test Tools4.9 GHz
Broadband Task Group
Video QualityPublic Safety Architecture Framework
Audio Quality ROW-B RF Propagation Studies
Modeling and Simulation
4
Video Quality
• Our goal is to develop recommendations for public safety practitioners– Minimum requirements to meet their
needs
• What is quality in public safety applications?– MOS is not appropriate for public safety– Video must be useful—we have taken a
task-based approach– Preliminary tests have been conducted
on the object recognition task
5
The Object Recognition Task
• The object recognition task is common across public safety applications– Different applications may have similar
requirements
• Scene content parameters:– Lighting conditions– Level of motion– Target size
• Scenes with equivalent values for these parameters form a “scenario group”
• Use cases with equivalent parameters for a “generalized use class”
6
The Test
• Quality is greatly impacted by the bitrate of compressed video
• We wish to understand the interaction between scene content parameters and bitrate
• Scenes from a variety of scenario groups with a variety of target objects were created, processed at different bitrates and presented to viewers who were asked to perform a recognition task
7
Test Details
• Scene content parameters– Lighting: Outdoor, bright, dim w/ flashing,
flashing only– Motion: Stationary, walking speed– Target Size: Small, large
• Items: Gun, taser, radio, flashlight, cell phone, mug, soda
• Processing parameters (HRC’s)– Resolutions: CIF, VGA– Bitrates: five choices for each resolution– All clips were encoded with H.264, baseline
profile
8
Lighting ExamplesOutdoor Bright
Dim w/ Flashing Flashing Only
9
Motion and Target Size ExamplesWalking, Small TargetWalking, Large Target
Stationary, Large Target
10
More Test Details
• For a given scenario group, scenes were held constant– Only the target object changes– Avoid memorization
• Methods described in ITU-T Recommendation P.912
• Collected data from 37 viewers, all public safety practitioners
11
Test Example
• Forced Choice
12
Results: LightingDaylight Bright
Flashing OnlyDim with Flashing
13
Results: Lighting
• Previous slide shows stationary, large target data
• We had hoped to be able to specify the bandwidth required for a given recognition level for each scenario group
• Saturation effect– Under poor lighting conditions, increasing
bandwidth does not always increase recognition performance
– Outdoor and bright indoor lighting are good enough, dim lighting is not
• CIF outperforms VGA in poor lighting– Easier to cope with flashing lights?
14
Results: Target Size
• These are outdoor, stationary data• Similar saturation effect
Large Target Small Target
15
Results: Motion – Large Target
• These are outdoor, large target data– Relatively small impact with large target
Stationary Walking Speed
16
Results: Motion – Small Target
• Much more significant impact with small target– Non-linear
• Motion also degrades performance in poor lighting
• VGA better than CIF
Stationary Walking Speed
17
Resolution
• CIF is better under poor lighting conditions
• VGA is better with motion• VGA is not significantly better than
CIF for a small target– Suggests CIF meets the minimum
resolution required to discern our small targets
– Clearly, further study would be beneficial
18
Conclusions
• Coding isn’t everything• More resolution isn’t always better• More work is required on automatic
classification of scenario groups• Under the right conditions, extremely
low bitrates can still be useful
19
Future Work
• Each of the scene parameters from this test should be studied separately, in depth
• There are additional parameters that determine a “generalized use class” (GUC) in the VQiPS user guide– Recorded vs. live video– Discrimination level
• e.g., general elements of the action, classification, positive identification
– We will explore these two dimensions in three phases of testing
20
Future Tests – Phase 1
• Very similar to previous test– Recorded video instead of live– Viewers will be allowed to pause the
video and step through frames – Viewers can replay each clip as many
times as desired before selecting an object
• Data is currently being gathered for this phase
21
Future Tests – Phases 2 and 3
• After Phase 1, we will have explored every dimension of the GUC’s except for discrimination level
• We will use the idea of acuity to extend our previous results along this dimension
22
Acuity
• Acuity is intended to be a one-dimensional measurement that describes the usefulness of a video system for recognition tasks
• Acuity will likely be measured in terms of a viewers ability to recognize characters on something similar to a Snellen chart.
• If we can map recognition rates to acuity and develop acuity requirements for each discrimination level, the data we have collected will allow us to make recommendations for every GUC.
23
Future Tests – Phase 2
• The purpose of this test is to measure the relationship between acuity and object recognition rates
• We will use video content similar to our previous tests, but charts for measuring acuity will be included in each scene.
• Viewers will be asked to recognize objects and read charts
24
Future Tests – Phase 3
• The purpose of this test is to measure acuity requirements for each discrimination level
• New video content for this test – Will include the acuity charts– Will be suitable for questions about various
levels of detail
• Viewers will be asked to read the charts• Viewers will be asked progressively more
difficult multiple choice questions about scene content– When a viewer answers incorrectly the test will
advance to the next scene
25
Video Quality in Public Safety (VQiPS)
• Bring people together– Police, fire fighters, emergency medical,
public transit, manufacturers, government, universities
• Learn what video quality means for public safety practitioners
• Express in technical terms – Requirements
• Common ground for different jobs– Application independent use classes
• Voluntary
26
VQiPS Motivations
Practitioners rely on video technology to keep people safe– Poor quality video quality can
mean the difference between life and death
Give practitioners the tools, support and information to make informed purchasing decisions
Unbiased guidance
First responders at wildfire scene. Tactical video can help incident controllers.
27
Video Quality in Public Safety (VQiPS)
• Outputs– User guide – Web tool– Glossary of terms– Find existing standards– Library of test video sequences
