g. boato, f. g. b. de natale, and p....
TRANSCRIPT
G. Boato, F. G. B. De Natale, and P. Zontone
VPQM 2010 - Jan. 13-15, 2010
Outline
Introduction
Digital forensics
New application: opinion mining
Example on contrast adjustment
Open issues
VPQM 2010 - Jan. 13-15, 2010 2
Introduction
3
AN INCONVENIENT TRUTH
bias? interests
behind information?
complete opinion
overview?
„global
warming“
Bias in the use of
images
VPQM 2010 - Jan. 13-15, 2010
Introduction
Web today
Social networking websites, image hosting websites, online
community platforms:
high number of digital image collections in the Web
important role of images and visual messages in the
communication process.
In this scenario..
modified data may influence people opinions and even alter their
attitudes in response to the represented event.
it is important to be able to automatically verify the fidelity and
authenticity of digital images in order to guarantee their
truthfulness.
VPQM 2010 - Jan. 13-15, 2010 4
5
Digital forensics to analyse modifications
which can impact on conveyed opinion
Introduction
VPQM 2010 - Jan. 13-15, 2010
Digital forensics
6VPQM 2010 - Jan. 13-15, 2010
Digital forensics
VPQM 2010 - Jan. 13-15, 2010 7
Digital forensics: goals
Image forgery detection
Image source identification
Discrimination between CG and real images
Was the picture captured using a digital camera?
Is it generated by computer graphics?
Which camera brand took this picture?
What processing has been done?
Has it been tampered or manipulated?
8VPQM 2010 - Jan. 13-15, 2010
Image forgeries detection
Pixel-based• Cloning
• Resampling
• Splicing
Format-based • JPEG Quantization
• Double Quantization
Camera-based• Chromatic aberration
• Color Filter Array
• Camera Response
Physics based • Light direction (2-D)
• Light direction (3-D)
Geometric-based• Principal Point
9VPQM 2010 - Jan. 13-15, 2010
Image source identification
Image formation in digital
camera and scanner
o Digital Camera Pipeline
o Scanner Pipeline
Source Model Identification o Image features
o CFA and Demosicing Artifacts
o Lens Distortion
Individual source identification o Imaging Sensor Imperfections
o Sensor Dust Characteristics
Success behind this technology:
Assumption that all images acquired by a single digital device have
particular intrinsic characteristics because of their image formation
pipeline and unique hardware components.
10VPQM 2010 - Jan. 13-15, 2010
CG or real?
11VPQM 2010 - Jan. 13-15, 2010
CG
CG
natural
natural
12VPQM 2010 - Jan. 13-15, 2010
Digital forensics & Opinion mining
New application of digital forensics for supporting
opinion mining on the Web:
analysis of the history of visual data
detection of manipulations which can impact on image
perception, therefore modifying the opinion conveyed by the
image.
Opinion mining on multimedia data to support
current tools working on text cross-media
characterization and analysis.
VPQM 2010 - Jan. 13-15, 2010 13
Digital forensics & Opinion mining
Image manipulations that can impact on human
perception:
Local modifications: local tampering, photomontage, etc.
14VPQM 2010 - Jan. 13-15, 2010
Digital forensics & Opinion mining
How local manipulations impact on the perception of
the information?
Image semantics required
Who was inserted through the photomontage?
What was hidden via copy & move?
Which CG object was inserted?
15VPQM 2010 - Jan. 13-15, 2010
Digital forensics & Opinion mining
Image manipulations that can impact on human
perception:
Global modifications: brightness, contrast, saturation, white
balance, levels, curves, hue, etc.
16VPQM 2010 - Jan. 13-15, 2010
Digital forensics & Opinion
mining Digital forensics usually focus on global processing
that not necessarily impact on perception and
opinion conveyed by the image
Specific techniques required
Current digital forensics techniques deal with resizing,
double JPEG compression, etc.
Deeper analysis of perceptual impact of modifications
New tools for CG / camera / scanner source identification
could be more appropriate and effective
17VPQM 2010 - Jan. 13-15, 2010
VPQM 2010 - Jan. 13-15, 2010 18
Image Quality Metrics (IQM)
Binary Similarity Measures (BSM)
Features extraction
Classifier SVM
(Support Vector Machines)
Training
Test
Original image
Contrast increase
Contrast reduction
Example: contrast adjustment
IQM features (11 values) properly support the detection of
statistics modifications both in the pixel and transform domain
(MSE, NCC, Spectral Magnitude Error, Angular Correlation
Measure, and Czekanowski Similarity Measure).
BSM features (270 values) are features computed on image
bit-planes which evidence differences in the levels of
decomposition. For our classification algorithm, bit-planes 3-8
have been considered for each RGB component.
An SVM classifier has been developed
Training: 200 images per class
Test: 100 images per class
VPQM 2010 - Jan. 13-15, 2010 19
Example: contrast adjustment
Contrast - Original Contrast +
Light contrast modifications
Heavy contrast modifications
Contrast - Original Contrast +
20VPQM 2010 - Jan. 13-15, 2010
Contrast +
Contrast –
Original
Total
contrast + (light)
contrast – (light)
contrast + (heavy)
contrast – (heavy)
21VPQM 2010 - Jan. 13-15, 2010
Accuracy results
Light modifications: total accuracy 80%
Contrast +
Contrast –
Original
Total
contrast + (light)
contrast – (light)
contrast + (heavy)
contrast – (heavy)
22VPQM 2010 - Jan. 13-15, 2010
Heavy modifications: total accuracy 93%
Accuracy results
VPQM 2010 - Jan. 13-15, 2010 23
Setup of a preliminary subjective experiment
25 images: five natural images modified according to the four
values of contrast corresponding to low/strong negative
adjustments and low/strong positive adjustments.
16 subjects: each subject was asked to classify the images
according to the feeling they felt they conveyed, not evaluating
the overall quality of each image.
Possible answers: very positive (5), positive (4), neutral (3),
negative (2), very negative (1).
Impact on opinion
VPQM 2010 - Jan. 13-15, 2010 24
Impact on opinion
VPQM 2010 - Jan. 13-15, 2010 25
Scores:
1 = very negative
2 = negative
3 = neutral
4 = positive
5 = very positive
Contrast adjustment impacts on opinion
perceived by the users.
Impact on opinion
Positive contrast variations A small increase in contrast can lead to a more positive
feeling conveyed by the image.
If the increase becomes too evident this implies a negative feeling due to the resulting artificial colors.
Negative contrast variations The feeling conveyed in this case is always worse than the
one conveyed by original unmodified images.
The new tool for contrast +/- adjustment detection can be exploited to infer opinion polarity.
VPQM 2010 - Jan. 13-15, 2010 26
Impact on opinion
VPQM 2010 - Jan. 13-15, 2010 27
Concluding remarks
Integration of multimedia data analysis in opinion
mining
Digital forensics for supporting opinion analysis on
digital images
Global and local modifications and new
requirements
The example of contrast adjustment
VPQM 2010 - Jan. 13-15, 2010 28
Open issues
Semantic analysis to support opinion the analysis of
local modifications
Identification of global modifications impacting on
opinion.
In which way quality and opinion are connected?
How can quality metrics be used for this kind of analysis?
How do image distortions impact on the opinion conveyed
by the image?
Design of new forensics tool
29
Questions?
VPQM 2010 - Jan. 13-15, 2010