4/11/2013

1

Image Tampering Detection

Yun Q. Shi and Patchara Sutthiwan

Intelligent Multimedia Lab

New Jersey Institute of Technologyshi@njit.edu24 April 2009

2

A Natural Image Model Approach to Tampering Detection

Acknowledgement

The contributions made by Professor Xuan, Drs. Zou, Fu, C. Chen, W. Chen, Su are deeply appreciated.

4/11/2013

2

3

Machine Learning Framework �A two- or multiple-class pattern recognition problem.

� yi = Rn is a feature vector and its associated class is ωj ∈{-

1,1}.ωi = 1 if yi is tampered image, otherwise ωj =-1.

� The classifier learns the mapping yi → ωj in the training stage and produces a decision function.

� Popular classifiers include Bayes, support vector machine (SVM) and neural network (NN).

Feature Extractor

PatternClassifier

InputImage

yiRecognized

Class ωj

4

Merits of Proposed Model

�Two combinations:• Combination of features derived from the image

pixel 2-D array, and those derived from the Multi-block DCT (MBDCT) coefficient 2-D arrays

• Combination of moments of characteristic functions based features and Markov process based features

�These features can make up each other when properly applied.

4/11/2013

3

5

Block Diagram of the Natural Image Model

Given Image(Pixel 2-D array)

MomentFeatures

•

MarkovFeatures

MarkovFeatures•

••

•••2×2 BDCT

Coefficient2-D Array

MomentFeatures

MarkovFeatures

•••

n×n BDCTCoefficient2-D Array

•••

N×N BDCTCoefficient2-D Array

2×2 BlockDiscrete Cosine

Transform•••

•••

N×N BlockDiscrete Cosine

Transform

n×n BlockDiscrete Cosine

Transform

•••

MarkovFeatures

MomentFeatures

•••

MarkovFeatures

MomentFeatures

•••

•••

MomentFeaturesMBDCT

6

2-DArray

Horizontal2-D Histogram

2-D DFT

•

Marginalmoments

•

•

•

Vertical2-D Histogram 2-D DFT

MarginalMoments

Main Diagonal2-D Histogram

2-D DFT MarginalMoments

Minor Diagonal2-D Histogram 2-D DFT Marginal

Moments

Prediction-error

2-D arrayGeneration

DFTDWT• Momentssubbands

LL0

Histogram

DFTDWT• Momentssubbands

LL0

Histogram

Moment Feature Generation

4/11/2013

4

7

HorizontalDifference2-D Array

(2T+1)2-DTransitionProbability

Matrix

Image Pixel2-D Array

orBDCT

Coefficient2-D Array

T

-T

•

•

•

VerticalDifference2-D Array

Main DiagonalDifference2-D Array

Minor DiagonalDifference2-D Array

(2T+1)2-DTransitionProbability

Matrix

(2T+1)2-DTransitionProbability

Matrix

(2T+1)2-DTransitionProbability

Matrix

•| |

T

-T

T

-T

T

-T

Markov Feature Generation

8

A Concrete Implementation� The main concern is the limited number of images in [1].� Block sizes are selected as 2×2, 4×4, and 8×8.

� For moment features, One-level Haar wavelet decomposition and horizontal and vertical 2-D histogramsonly.

� When computing Markov features, we only apply Markov process to horizontal and vertical difference 2-D array of 8×8 BDCT coefficient 2-D array and threshold T = 4.

� Feature dimensionality: 168 + 162 = 330.

� SVM is employed as the classifier.

4/11/2013

5

9

Given Image(Pixel 2-D array)

Moment FeatureExtraction

•

42-D

2×2 BDCTCoefficient2-D Array

4×4 BDCTCoefficient2-D Array

8×8 BDCTCoefficient2-D Array

8×8 BlockDiscrete Cosine

Transform

4×4 BlockDiscrete Cosine

Transform

2×2 BlockDiscrete Cosine

Transform

Moment FeatureExtraction

42-D

Moment FeatureExtraction

42-D

Moment FeatureExtraction

42-D

Markov FeatureExtraction

162-D

Block Diagram of the Implementation

1010

Examples of spliced images (Columbia dataset)

4/11/2013

6

1111

1212

4/11/2013

7

1313

Experimental results on Columbia dataset (I)

Dartmouth Columbia NJIT-1 NJIT-2 NJIT-3

73% 71% 80% 82% 91%

On Columbia Dataset

14

10 randomly selected authentic images from Columbia dataset

14

4/11/2013

8

15

10 randomly selected spliced images from Columbia dataset

15

1616

• Kerry and Fonda (2004) → success• Israel air raid Lebanon (2006) → success• Iraqi solder (2003) → half success half

failure

Experimental results on real image tampering cases (II)

three well-known news examples

4/11/2013

9

17

Kerry and Fonda (2004)

17

Picture Actual type Detected as authentic

Detected as tampered

Kerry Authentic 20/20 0/20

Fonda Authentic 15/20 5/20

Kerry & Fonda Tampered 0/20 20/20

18

Israel Air Raid Lebanon (2006)

18

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 20/20 0/20

On the right Tampered 0/20 20/20

4/11/2013

10

19

US Soldier in Iraqi (2003)

19

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 0/20 20/20

In the middle Authentic 0/20 20/20

On the right Tampered 0/20 20/20

2020

• Girl (Air Force Research Lab) → success• Stalin → success• April 09, Israel → success• ABC News lady → success• Rove → success• Fire on Ice → success• Martha, Newsweek → success• Pitt and Julia → success• Reagan → success

Experimental results on real image tampering cases (III)

4/11/2013

11

21

Girl (Air Force Research Lab)

21

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 13/20 7/20

On the right Tampered 2/20 18/20

22

Stalin

22

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 19/20 1/20

On the right Tampered 7/20 13/20

4/11/2013

12

2323

April 09, Israel

24

April 09, Israel

24

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 15/20 5/20

On the right Tampered 0/20 20/20

4/11/2013

13

2525

ABC News Lady

Actualtype

Tampered

Detected as authentic

7/20

Detected as tampered

13/20

26

Karl Rove

26

4/11/2013

14

2727

Karl Rove

Actualtype

Tampered

Detected as authentic

0/20

Detected as tampered

20/20

28

Fire on Ice

28

Actualtype

Tampered

Detected as authentic

0/20

Detected as tampered

20/20

4/11/2013

15

29

Martha

29

Actualtype

Tampered

Detected as authentic

0/20

Detected as tampered

20/20

30

Brad and Angelina

30

4/11/2013

16

31

Brad and Angelina

31

Actualtype

Tampered

Detected as authentic

0/20

Detected as tampered

20/20

32

Reagan

32

Picture Actual type Detected as authentic

Detected as tampered

On the left Tampered 6/20 14/20

On the right Tampered 1/20 19/20

4/11/2013

17

3333

• Tibet Antelope → success• Hitler → failure (analogue, other processing) • Iranian missile → failure (size, other factor) • Poster and soldiers → success (size)

• Nonsufficient training• Un-known factors

– Head-shoulder (some success, some failure)

Experimental results on real image tampering cases (IV)

34

Tibet AntelopeFeb. 25, 2008 (Chinese News Agency)

34

4/11/2013

18

35

Tibet AntelopeFeb. 25, 2008 (Chinese News Agency)

35

Actualtype

Tampered

Detected as authentic

2/20

Detected as tampered

18/20

36

Hitler

36

→

4/11/2013

19

37

Hitler

37

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 4/20 16/20

On the right Tampered 20/20 0/20

38

Iranian Missiles

38

4/11/2013

20

39

Iranian Missiles

39

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 0/20 20/20

On the right Tampered 0/20 20/20

40

Poster and Soldier

40

4/11/2013

21

41

Poster and Soldier

41

Actualtype

Tampered

Detected as authentic

0/20

Detected as tampered

20/20

42

Experimental results on real image tampering cases (V)

Two images from Nick

1st: Daisy

4/11/2013

22

43

2nd: Daisy and Ivy

4444

Picture Actual type Detected as authentic

Detected as tampered

On the left Authentic 20/20 0/20

On the right Tampered 0/20 20/20

4/11/2013

23

4545

4646

Picture Actual type Detected as authentic

Detected as tampered

On the left Tampered 0/20 20/20

On the right Tampered 7/20 13/20

4/11/2013

24

4747

Classification Results from Columbia Dataset

Filename Type Detected as authentic Detected as tampered

blk7 authentic 20/20 0/20

blk36 authentic 20/20 0/20

blk_3_63 authentic 20/20 0/20

blk_4_46 authentic 20/20 0/20

blk_5_28 authentic 20/20 0/20

blk_6_1 authentic 20/20 0/20

blk_8_74 authentic 20/20 0/20

blk_9_12 authentic 20/20 0/20

blk_10_21 authentic 20/20 0/20

blk_11_44 authentic 20/20 0/20

4848

blk_7_s tampered 12 8

blk_12_20 tampered 0 20

blk_13_33 tampered 0 20

blk_14_26 tampered 0 20

blk_14_58 tampered 0 20

blk_14_88 tampered 0 20

blk_14_106 tampered 0 20

blk_16_26 tampered 0 20

blk_17_4 tampered 4 16

blk_17_91 tampered 0 20

4/11/2013

25

4949

Classification Results from Real Image Tampering Cases

Description Filename

Actual

Type

Detected as

Authentic by

(per 20

classifiers)

Detected as

Tempered/Spliced

by (per 20

classifieres)

Fonda 4a.bmp authentic 15 5

Kerry 5a.bmp authentic 20 0

Fonda and Kerry

Combined 6t.bmp tampered 0 20

Israel air raid

Lebanon 7a.bmp authentic 20 0

8t.bmp tampered 0 20

Tibet Goat

tibetgoat_crop

ped.bmp tampered 2 18

Stalin (cropped

with 256x235) stalin_a.bmp authentic 19 1

stalin_t.bmp tampered 7 13

5050

April 09, Israel y_a.bmp authentic 15 5

y_t.bmp tampered 0 20

solder (cropped

with 256x256 at

the messages) solder_t.bmp tampered 0 20

Martha on

Newsweek cover martha_t.bmp tampered 0 20

ABC News abc_news_t.bmp tampered 7 13

Rove rovekarl_t.bmp tampered 7 20

April 09, Detrich's

Photo, ball added

basketball_small_t.b

mp tampered 0 20

Mussolini mussolini_a.bmp authentic 14 6

mussolini_t.bmp tampered 20 0

Black girl IF9905_5.jpg authentic 13 7

IF9905_7.jpg tampered 2 18

4/11/2013

26

5151

fire on ice fireonice_t.bmp tampered 0 20

Pitt (cropped with

193x132) pitt_t.bmp tampered 0 20

Regan on Times

(200x244 full size) reagan.jpg tampered 6 14

(cropped with

128x192) reagan_mod_t.bmp tampered 1 19

Iran Missile iran_missile_a.bmp authentic 0 20

iran_missile_t.bmp tampered 0 20

Hitler hitler_a.bmp authentic 4 16

hitler_t.bmp tampered 20 0

Lena IF9905_9.jpg authentic 20 0

IF9905_10.jpg tampered 20 0

Castro

castro_a.bmp

(256x256) authentic 18 2

castro_t.bmp

(256x256) tampered 10 10

castro_t2.bmp

(368x278) tampered 16 4

castro_t3.bmp

(256x256) tampered 18 2

5252

� Data set of authentic and spliced image blocks, DVMM, Columbia University, http://www.ee.columbia.edu/dvmm/researchProjects/AuthenticationWatermarking/BlindImageVideoForensic/

� http://www.ee.columbia.edu/dvmm/publications/04/TR_splicingDataSet_ttng.pdf� H. Farid: High-order statistics and their applications to digital forensics, IEEE Workshop

on Statistical Analysis in Computer Vision (in conjunction with CVPR), 2003. � T.-T. Ng, S.-F. Chang and Q. Sun, “Blind detection of photomontage using higher order

statistics”, IEEE ISCAS04, May, 2004.� D. Fu, Y. Q. Shi, and W. Su: Detection of image splicing based on Hilbert-Huang

transform and moments of characteristic functions with wavelet decomposition, IWDW2006.

� W. Chen, Y. Q. Shi, and W. Su: Image splicing detection using 2-D phase congruency and statistical moments of characteristic function, SPIE2007.

� Y. Q. Shi, C. Chen, W. Chen, “A natural image model approach to splicing detection,” ACM Workshop on Multimedia Security, Dallas, Texas, September 2007.

References