Detecting malicious tampering in digital images

Markos Zampoglou - markzampoglou@iti.gr

Information Technologies Institute (ITI)Centre for Research and Technology Hellas (CERTH)

Workshop on Tools for Video Discovery & Verification in Social MediaDec 14, 2017 @ Thessaloniki, Greece

Image forgeries in the news

https://www.snopes.com/photos/animals/puertorico.asp

Image forgeries in the news

http://www.imperfectedblog.com/2015/11/crimes-of-retouching-and-the-importance-of-community/

Image forgeries in the news

https://thelede.blogs.nytimes.com/2008/07/10/in-an-iranian-image-a-missile-too-many/

The promise of image forensics

Image verification: tools of the trade

• Metadata analysis• Photoshop? Dates/locations? Copyrights?

• Reverse image search• Content analysis / tampering localization

Tampering localization algorithms

Splicing localization

Copy-move detection

Block matching

Keypoint matching

High-frequency noise

CFA patterns

JPEG compression traces

Filtering and analysis

Camera-specific (PRNU)

Quantization artefacts

Compression grid misalignment

JPEG ghosts, ELA

REVEAL Media Verification Assistant (1/2)

http://reveal-mklab.iti.gr/

• REVEAL (FP7, 2013-2016)

• Second round of testing as part of InVID

• Collaboration between CERTH and DW

• Features:• Metadata: listing, GPS geolocation, EXIF thumbnails

• Reverse image search: Google Images integration

• Tampering detection: Eight state-of-the-art algorithms

• Aims:• a comprehensive, self-contained verification toolset

• an evaluation framework for verification tools and protocols

REVEAL Media Verification Assistant (2/2)

Zampoglou, M., Papadopoulos, S., Kompatsiaris, Y., Bouwmeester, R., & Spangenberg, J. (2016, April). Web and Social Media Image Forensics for News Professionals. In SMN@ ICWSM.

Comparison of verification tools

FotoForensics1 Forensically2 Ghiro3 REVEAL4

ELA X X X X

Ghost X

DW Noise X

Median Noise X X

Block Artifact X

Double Quantization X

Copy-move X*

Thumbnail X X

Metadata X X X X

Geotagging X X X X

Reverse search X

1http://fotoforensics.com2http://29a.ch/photo-forensics/3http://www.imageforensic.org/4http://reveal-mklab.iti.gr/reveal/verify.html

*Forensically implements a very simple block-matching algorithm with low robustness

Further features

• Open source1

• Public demo version• Real-world examples of detections and non-

detections• Tutorials and algorithm explanations• “Magnifying glass” for examining fine details• Scrolling image/map overlay • Export personalized analysis as PDF

1https://github.com/MKLab-ITI/image-forensics

Usage distribution

France 401Netherlands 262Germany 214UK 181US 153Argentina 96Egypt 52

Current challenges (1/2)

• Absence of automatic methods to analyse metadata• High computational requirements• Interpretation of results not always intuitive

Current challenges (2/2)

• When applied in real-world cases, algorithms perform significantly worse than in research datasets• This could partly be attributed to the effects of Social

Media dissemination

Identifying Untampered Images (1/2)

• Interpreting the results (especially for non-detections) can be an issue

Untampered:

Tampered:

Algorithm: ADQ1 (Lin et al, 2009)

Identifying Untampered Images (2/2)

• Not all algorithms are created equal• Still, some training is usually necessary

Untampered:

Tampered:

Algorithm: ADQ2 (Bianchi et al, 2011)

The pitfalls of image forensics

“MH17 – Forensic Analysis of Satellite Images Released by the Russian Ministry of Defence”

• A report by Bellingcat1 using ELA to prove that the Russian MoD had forged the images

• N. Krawetz: Bellingcat “misinterpreted the results”2

1https://www.bellingcat.com2http://www.hackerfactor.com/blog/index.php?/archives/676-Continuing-Education.html

Future steps

• Add a tampering probability score• Generate a “merged” tampering heat map• Test new approaches based on deep learning

• Keep improving stability, speed, GUI responsiveness

Acknowledgements

• Dr. Chryssanthi Iakovidou (CAGI algorithm)

• Olga Papadopoulou (back-end development)

• Lazaros Apostolidis (front-end development)

• Dr. Symeon Papadopoulos (Overview & design)

• Ruben Bouwmeester, DW (Design and evaluation)

• Yiannis Kompatsiaris (Overview)

Thank you!

http://reveal-mklab.iti.gr/reveal

Get in touch!

Markos Zampoglou markzampoglou@iti.gr

Symeon Papadopoulos papadop@iti.gr

References

• Bianchi, Tiziano, Alessia De Rosa, and Alessandro Piva. "Improved DCT coefficient analysis for forgery localization in JPEG images." In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 2444-2447. IEEE, 2011.

• Ferrara, Pasquale, Tiziano Bianchi, Alessia De Rosa, and Alessandro Piva. "Image forgery localization via fine-grained analysis of cfa artifacts." Information Forensics and Security, IEEE Transactions on 7, no. 5 (2012): 1566-1577.

• Farid, Hany. "Exposing digital forgeries from JPEG ghosts." Information Forensics and Security, IEEE Transactions on 4, no. 1 (2009): 154-160.

• Fontani, Marco, Tiziano Bianchi, Alessia De Rosa, Alessandro Piva, and Mauro Barni. "A framework for decision fusion in image forensics based on dempster–shafer theory of evidence." Information Forensics and Security, IEEE Transactions on 8, no. 4 (2013): 593-607.

• Lin, Zhouchen, Junfeng He, Xiaoou Tang, and Chi-Keung Tang. "Fast, automatic and fine-grained tampered JPEG image detection via DCT coefficient analysis." Pattern Recognition 42, no. 11 (2009): 2492-2501.

• Mahdian, Babak and Stanislav Saic, “Using noise inconsistencies for blind image forensics,” Image and Vision Computing, vol. 27, no. 10, pp. 1497–1503, 2009.

• Zampoglou, Markos, Symeon Papadopoulos, and Yiannis Kompatsiaris. "Detecting image splicing in the wild (WEB)." In Multimedia & Expo Workshops (ICMEW), 2015 IEEE International Conference on, pp. 1-6. IEEE, 2015.

Evaluation methodology

• Quantitative• Six reference datasets (images + binary masks of tampering

= “ground truth”)• Measures capturing the matching between ground truth

mask and algorithm output• Comparison of 14 algorithms, “best” six plus a newly

proposed one ended up in the tool

• Qualitative• Informal feedback has been received by end users• Pertains to both usability and quality of results

Zampoglou, M., Papadopoulos, S., & Kompatsiaris, Y. (2017). Large-scale evaluation of splicing localization algorithms for web images. Multimedia Tools and Applications, 76(4), 4801-4834.

Quantitative evaluations (1/3)

• To assess the performance of the implemented algorithms, we evaluated their performance on a large number of images

• Benchmark datasets:• Columbia Uncompressed (Hsu and Chang 2006, “COLUMB”), First

Image Forensics Challenge (“CHAL”)1, Fontani et al 2013 (“FON”)

• Additional datasets• The Wild Web Dataset (Zampoglou et al, 2015)

• The Deutsche Welle Image Forensics Dataset

1http://ifc.recod.ic.unicamp.br/fc

Quantitative evaluations (2/3)

• Evaluations on benchmark datasets:

• Robustness to resaves:

Quantitative evaluations (3/3)

• Performance on the Wild Web Dataset:

• Performance on the Deutsche Welle Image Forensics Dataset:• Successful detection for 2 out of 6 cases (4 out of 12

images)

Algorithm DQ GHO BLK ELA DWHF MED

Detections(out of 80)

3 12 3 2 3 1

Time (sec) 0.27 6.12 13.40 1.29 122.13 0.54

Image verification software: requirements• Easy to use; requires little expertise and training• Integrates easily in established workflows and IT

systems• Provides all common functionalities

• Metadata analysis• Reverse image search• Tampering localization

• Clear, interpretable and transparent output• No “black box” results

• Intuitive and simple interface• Allows sharing and archiving of reports and

conclusions• Comparable or better results than existing solutions