Exploiting Multi-domain Visual Informationfor Fake News Detection

Peng Qi1,2, Juan Cao1,2, Tianyun Yang1,2, Junbo Guo1 and Jintao Li11Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

2University of Chinese Academy of Sciences, Beijing, China{qipeng,caojuan,yangtianyun,guojunbo,jtli}@ict.ac.cn

Abstract—The increasing popularity of social media promotesthe proliferation of fake news. With the development of multime-dia technology, fake news attempts to utilize multimedia contentswith images or videos to attract and mislead readers for rapiddissemination, which makes visual contents an important part offake news. Fake-news images, images attached in fake news posts,include not only fake images which are maliciously tampered butalso real images which are wrongly used to represent irrelevantevents. Hence, how to fully exploit the inherent characteristicsof fake-news images is an important but challenging problemfor fake news detection. In the real world, fake-news imagesmay have significantly different characteristics from real-newsimages at both physical and semantic levels, which can be clearlyreflected in the frequency and pixel domain, respectively. There-fore, we propose a novel framework Multi-domain Visual NeuralNetwork (MVNN) to fuse the visual information of frequency andpixel domains for detecting fake news. Specifically, we designa CNN-based network to automatically capture the complexpatterns of fake-news images in the frequency domain; andutilize a multi-branch CNN-RNN model to extract visual featuresfrom different semantic levels in the pixel domain. An attentionmechanism is utilized to fuse the feature representations offrequency and pixel domains dynamically. Extensive experimentsconducted on a real-world dataset demonstrate that MVNNoutperforms existing methods with at least 9.2% in accuracy,and can help improve the performance of multimodal fake newsdetection by over 5.2%.

Index Terms—fake news detection, fake-news images, multi-domain, social media

I. INTRODUCTION

Social media has become a major information platform,where people acquire the latest news and express their opinionsfreely. However, the convenience and openness of socialmedia have also promoted the proliferation of fake news,which caused many detrimental societal effects. For instance,during the month before the 2016 U.S. presidential electioncampaign, the American encountered between one and threefake stories in average from known publishers [1], whichinevitably misled the voters and influenced the election results.Hence, the automatic detection of fake news has become anurgent problem of great concern in recent years [2]–[5].

The development of multimedia technology promotes theevolution of self-media news from text-based posts to mul-timedia posts with images or videos, which provide betterstorytelling and attract more attention from readers. It isestimated that the average number of reposts for posts withimages is 11 times larger than those without images [6].

Original ImageTampered Image

(a) Tampered Images

(b) Misleading Images

Fig. 1: Examples of fake-news images: (a) A tampered imagewhere Putin is spliced on the middle seat at G-20 to showthat he is in the middle of an intense discussion between otherworld leaders. (b) Three misleading images: the left one is anartwork which is reposted as a photo of solar eclipse (March2015); the middle one is a real image captured in 2009 NewYork plane crash, but it is claimed to be the wrecked MalaysiaAirlines MH370 in 2014; the right one is a real image used tomisinterpret Trump’s behavior while he is just imitating Abe.

Unfortunately, this advantage is also taken by fake news whichusually contains misrepresented or even tampered images toattract and mislead readers for rapid dissemination. As a result,visual content has become an important part of fake news thatcannot be neglected.

According to existing research [7], fake-news images, im-ages attached in fake news posts, include not only fake imageswhich are maliciously tampered but also real images whichare wrongly used to represent irrelevant events. In detail,we broadly classify fake-news images into two categories:tampered images and misleading images. Tampered imagesmean fake-news images which have been digitally modifiedas Fig. 1a shows, equal to fake images in our common sense.Misleading images refer to fake-news images that haven’texperienced any manipulation, but the content is misleading asdescribed in Fig. 1b. These misleading images usually derivefrom artworks or outdated images which are published on anearly event [8]. Existing researches on fake news detectionmainly focus on text content [9]–[12] and social context[13], [14], while a few works utilize visual information to

arX

iv:1

908.

0447

2v1

[cs

.MM

] 1

3 A

ug 2

019

Conv 2D

Conv 2D

MaxPooling

Conv 2D

Conv 2D

MaxPooling

Conv 2D

FC

GRU

GRU

Conv 1D

MaxPooling FC

Frequency

Domain

Sub-

network

Pixel

Domain

Sub-

network

Fusion

Sub-

network

ATTENTION

…

𝐻0

𝐻1

𝐻63

Binary Classifier

FCConv

1DMax

Pooling

Conv 1D

MaxPooling

Conv 2D

Conv 2D

MaxPooling

Conv 2D

Conv 2D

MaxPooling

Conv 2D

FC

GRU

GRU

Conv 2D

FC

GRU

GRU

Conv 2D

FC

GRU

GRU

Fig. 2: An illustration of the proposed Multi-domain Visual Neural Network (MVNN). It mainly consists of three components:a frequency domain sub-network, a pixel domain sub-network, and a fusion sub-network. The frequency domain sub-networkfirst transforms the input image from pixel domain to frequency domain, and utilizes a CNN-based model to capture thephysical characteristics of this image. The pixel domain sub-network employs a multi-branch CNN-RNN network to extractthe features of different semantic levels of the input image. The fusion sub-network dynamically fuses feature vectors obtainedfrom the frequency and pixel domain sub-network through an attention mechanism to classify the input image as a fake-newsor real-news image.

(a) A fake-news image (b) A real-news image

Fig. 3: Comparison of fake-news images and real-news imagesat the physical level. We observe that the fake-news image(a) has obvious block effect while the real-news image (b) ismore clear. We enlarge the faces of the two images for bettercomparison.

detect fake news. Some works [7], [15] extract forensicsfeatures to evaluate the authority of attached images, but theseforensics features are mostly crafted manually for detectingspecific manipulated traces, which are not applicable formisleading images. Other works [16]–[18] utilize pre-trainedconvolutional neural network like VGG19 [19] to obtaingeneral visual representations, which can hardly capture thesemantic commonalities of fake-news images due to the lack oftask-relevant information. Therefore, how to fully exploit theinherent characteristics of fake-news images is an importantbut challenging problem for distinguishing fake news from realnews by utilizing visual contents.

In the real world, fake-news images may have significantly

(a) (b) (c)

Fig. 4: Comparison of fake-news images and real-news imagesat the semantic level. We can find that fake-news images aremore visually striking and emotional provocative than real-news images, even though they describe the same type of eventsuch as fire (a), earthquake (b) and road collapse (c).

different characteristics from real-news images at both physi-cal and semantic levels:

At the physical level, fake-news images might be oflow quality, which can be clearly reflected in the frequencydomain. For example, after being uploaded to and down-loaded from social platforms multiple times, misleading im-ages usually have heavier re-compression artifacts than real-news images such as block effect as shown in Fig. 3. Besides,there are manipulation traces in tampered images inevitably.Considering that re-compressed and tampered images often

present periodicity in the frequency domain, which can beeasily characterized by CNN which has the ability to capturespatial structure features, we design a CNN-based network toautomatically capture the characteristics of fake-news imagesin the frequency domain (see the top part of Fig. 2).

At the semantic level, fake-news images also exhibit somedistinct characteristics in the pixel domain (also known asspatial domain). Fake news publishers tend to utilize images toattract and mislead readers for rapid dissemination; thus fake-news images often show visual impacts [20] and emotionalprovocations [2], [21] just as Fig. 4 shows. These character-istics have been proven related to many visual factors fromlow-level to high-level [22]; thus we build a multi-branchCNN-RNN network to extract features of different semanticlevels (see the bottom part of Fig. 2), for fully capturing thecharacteristics of fake-news images in the pixel domain.

We have shown that the visual features at both physical andsemantic levels are important for detecting fake-news images;thus fusing the visual information of the frequency and pixeldomains has the potential to improve the performance of fakenews detection. Intuitively, not all features contribute equallyto the task of fake news detection, which means some visualfeatures play more important roles than others in evaluatingwhether a given image is fake-news or real-news image.Therefore, we employ an attention mechanism [23] to fusethese visual features from different domains dynamically.

To sum up, we propose a Multi-domain Visual NeuralNetwork (MVNN) framework (see Fig. 2), which can learneffective visual representations by combining the informationof frequency and pixel domains for fake news detection. Theproposed model consists of three main components: a fre-quency domain sub-network and a pixel domain sub-networkused to capture the characteristics of fake-news images atphysical and semantic levels respectively, and a fusion sub-network to fuse these features dynamically. In summary, thecontributions of this paper are three folds:

1) To the best of our knowledge, we are the first to utilizemulti-domain visual information for fake news detection,which captures the inherent characteristics of fake-newsimages at both physical and semantic levels.

2) We propose a novel framework MVNN, which exploitsan end-to-end neural network to learn representationsof frequency and pixel domains simultaneously andeffectively fuse them.

3) We conduct extensive experiments on a real-worlddataset to validate the effectiveness of the proposedmodel. The results demonstrate that our model is muchbetter than existing methods, and the visual represen-tations learned by our model can help improve theperformance of multimodal fake news detection by alarge margin. Besides, we show that the informationof frequency and pixel domains are complementary fordetecting fake news.

II. RELATED WORK

In the task of fake news detection, the main challenge ishow to utilize information from different modalities to distin-guish fake news posts and real news posts. Most of existingapproaches focus on text content [9]–[12] and social context[13], [14] which refers to information generated during thenews propagation process on social networks. Recently, visualinformation has been shown to be an important indicator forfake news detection [2], [6]. With the popularity of multimediacontent, researchers begin to incorporate visual information todetect fake news.

Some early works use basic statistical features about at-tached images to help classify fake news posts, such as thenumber of attached images [10], [24], image popularity andimage type [6]. However, these statistical features are so basicthat they can hardly represent complex distributions of visualcontents in fake news.

Visual forensics features are generally used for imagemanipulation detection. To evaluate the authority of attachedimages, some works extract visual forensics features, such asblock artifact grids (BAG), to assist fake news detection. Forexample, the Verifying Multimedia Use task on 2015 [25] and2016 MediaEval benchmark [7] provide seven types of imageforensics features to help detect manipulated and misleadinguse of web multimedia content. Based on these providedforensics features, [15] extract advanced forensics featuresand combine them with post-based and user-based featuresin a semi-supervised learning scheme to tackle the newsverification problem. However, most of these forensics featuresare crafted manually for detecting specific manipulated traces,which are not applicable for the real images attached in fakenews. Besides, these hand-crafted features are labor-expensiveand limited to learn complicated patterns, causing the poorgeneralization performance on the task of fake news detection.

Inspired by the power of CNN, most existing works basedon multimedia contents use a pre-trained deep CNN likeVGG19 [19] to obtain general visual representations and fusethem with text information. Specifically, [16] first incorporatesmultimodal contents on social networks via deep neural net-works to solve fake news detection problem; [17] proposes anend-to-end event adversarial neural network to detect newly-emerged fake news events based on multi-modal features; [18]presents a novel approach to learn a shared representationof multimodal information for fake news detection. However,these works focus on how to fuse the information of differ-ent modalities, ignoring effectively modeling visual contents.These visual features they adopted are too general to reflect theintrinsic characteristics of fake-news images due to the lackof task-relevant information, which degrades the performanceof visual contents in fake news detection.

To overcome these limitations of existing works, we proposea novel deep learning network to model visual contents atboth physical and semantic levels for the task of fake newsdetection.

III. PROBLEM FORMULATION

Fake news is defined as news articles that are intentionallyand verifiably false and could mislead readers [26], which hasbeen widely adopted in recent studies [2]. Different from thetraditional definition, in the context of microblog, fake newsaims at news posts that are published on social media by userswhich are usually less than 140 characters instead of newsarticles. Formally, we state this definition as follows,

Definition 1: Fake news: In the context of microblog, a pieceof fake news is a news post that is intentionally and verifiablyfalse.

Definition 2: Fake-news images: A fake-news image is animage attached in fake news.

The problem addressed in this paper is how to utilize thevisual content to identify a news post as real or fake, whichequals to classify a given image as fake-news image or real-news image. Consequently, we formally definite the studiedproblem as follows:

Problem 1: Given a set of news posts X ={x1, x2, . . . , xm}, corresponding images I = {i1, i2, . . . , im},and labels Y = {y1, y2, . . . , ym}, learn a classifier f that canutilize the corresponding image to classify whether a givenpost is fake news (yt = 1) or real news (yt = 0) , i.e.,yt = f(it).

IV. METHODOLOGY

In this section, we first make an overview of the proposedMulti-domain Visual Neural Network (MVNN), and thenintroduce the three major components in detail.

A. Model Overview

The goal of the proposed MVNN model is utilizing thevisual information in frequency and pixel domains to evaluatewhether the given image is a fake-news or real-news image.As shown in Fig. 2, MVNN includes three major components:a frequency domain sub-network, a pixel domain sub-networkand a fusion sub-network. For an input image, we feed it intothe frequency and pixel domain sub-network to obtain featuresat physical and semantic levels respectively. The fusion sub-network takes these learned features as input and learns afinal visual representation of this image to predict whetherthis image is a fake-news or real-news image.

B. Frequency Domain Sub-network

The physical features of images can reflect their originalityto a certain extent which is helpful for detecting fake news;thus we design the frequency domain sub-network to extractfeatures of the input image at the physical level. Accordingto existing studies about image forensics [27]–[30], discretecosine transform (DCT) has been widely utilized for capturingthe tampered and re-compressed architects, so we employDCT on the input image to transform it from pixel domain tofrequency domain. Considering that tampered images and re-compressed images often present periodicity in the frequencydomain, which can be easily characterized by CNN with theability to capture spatial structure characteristics; thus we

Conv 1D3,32

MaxPooling

2FC

…

𝐻0

𝐻1

𝐻63

DCT

1-D Fourier Transform

(250,1)Sampling

FC

𝑤0

𝑤1

𝑤63

𝑙0Conv 1D

3,64

MaxPooling

2

Conv 1D3,128

MaxPooling

2

Fig. 5: The detailed architecture of the frequency domainsub-network. For an input image, we first transform it frompixel domain to frequency domain, and design a CNN-basednetwork to obtain its feature representation in the frequencydomain.

design a CNN-based network to capture the characteristicsof fake-news images in the frequency domain. The detailedarchitecture of this part is shown in Fig. 5.

For an input image, we first employ block DCT on it toobtain 64 histograms of DCT coefficients corresponding to 64frequencies. Following the process of [28], we then carry 1-D Fourier transform on these DCT coefficient histograms toenhance the effect of CNN. Considering that CNN needs aninput of a fixed size, we sample these histograms and obtain64 250-dimensional vectors, which can be represented as{H0, H1, ...H63}. After being pre-processed, each input vectoris fed to the shared CNN network to obtain correspondingfeature representation {w0, w1, ...w63}. The CNN networkconsists of three convolutional blocks and a fully connectedlayer, and each convolutional block is composed of a one-dimensional convolutional layer followed by a max-poolinglayer. To accelerate the convergence of the model, we set thenumber of filters in the convolution layers to be incremental.Existing works about image forensics usually only considercoefficients of a part of frequencies. However, we find that allfrequencies contribute to the task of fake news detection, hencewe fuse the feature vectors of all frequencies by concatenatingand feed the final feature representation l0 to the fusion sub-network. Specifically, we experiment with different fusingmethods, and results show that concatenating performs bestin this task.

C. Pixel Domain Sub-network

We have shown that fake-news images have different char-acteristics from real-news images at the semantic level, indicat-ing that the semantic features are important in detecting fakenews. Therefore, we design the pixel domain sub-network forextracting visual features of the input image at the semanticlevel, of which the detailed architecture is shown in Fig. 6.

CNN learns high-level semantic representations throughlayer-by-layer abstraction from local to global view, of whichthe earlier layers prefer low-level features such as color, lineand shape, while the later layers tend to focus on the objects.In the process of abstraction, low-level features will inevitablysuffer some losses, which further suggests that the bottomand intermediate layer in CNN can provide complementaryinformation for the top layer. Many works have proven inte-grating features from different layers can help achieve betterperformance than only using high-level features for sometasks such as salient object detection and image emotion

classification [31]–[33]. We have shown that fake-news imagesusually show some visual impacts and emotion provocation,which have been proven related to many visual factors fromlow-level to high-level [22]. Therefore, to fully capture thesemantic characteristics of fake-news images, we build a multi-branch CNN network to extract multiple levels of features andutilize a bidirectional GRU (Bi-GRU) network to model thesequential dependencies between these features.

As Fig. 6 described, the CNN network mainly consists offour blocks composed of a 3× 3 and 1× 1 convolution layerand a max-pooling layer. The input image is fed to the CNN,and features extracted from four branches will pass through a1×1 convolutional layer and a fully connected layer, obtainingthe corresponding feature vectors vt, t ∈ [1, 4] . These featuresrepresent different parts of images, such as line, color, texture,and object, which characterize different levels of features fromlocal to global view. Motivated by Inception module used inGoogLeNet [34], we use 1 × 1 convolution layer to reducethe dimension and increase the model’s representative powerbecause it increases the non-linear activation and promotes theinformation fusion of different channels.

Intuitively, there are strong dependencies between differentlevels of features. For example, middle-level features such astextures, consist of low-level features such as lines, and mean-while compose high-level features such as objects. Therefore,we utilize GRU to model these dependencies between low-level and high-level features. Specifically, we model these fea-tures from different levels as a sequence V = {vt}, t ∈ [1, 4],where vt represents the visual features extracted from t-thbranch in the CNN network described in Fig. 6. Then theentire pipeline in GRU at time step t can be presented asfollows:

rt = σ(Wr[vt, ht−1] + br) (1)

zt = σ(Wz[vt, ht−1] + bz) (2)

ht = tanh(Wh[vt, rt � ht−1] + bh) (3)

ht = (1− zt)� ht−1 + zt � ht (4)

where rt, zt, ht, ht are the reset gate, update gate, hiddencandidate, and hidden state respectively. W are the weightmatrices and b are the bias terms. Besides, σ stands for thesigmoid function and � represents the element-wise multipli-cation.

Conv 2D3*3,32

Conv 2D1*1,32

MaxPooling

2*2

Conv 2D3*3,64

Conv 2D1*1,64

MaxPooling

2*2

Conv 2D3*3,64

Conv 2D1*1,64

MaxPooling

2*2

Conv 2D3*3,128

Conv 2D1*1,128

MaxPooling

2*2

Conv 2D1*1,64

FC

GRU

GRU

Conv 2D1*1,64

FC

GRU

GRU

Conv 2D1*1,64

FC

GRU

GRU

Conv 2D1*1,64

FC

GRU

GRU

(224,224,3)𝑣1 𝑣2 𝑣3 𝑣4

𝑙1 𝑙2 𝑙3 𝑙4

Branch1 Branch2 Branch3 Branch4

Fig. 6: The detailed architecture of the pixel domain sub-network. For an input image, we utilize a multi-branch CNN-RNN network to extract its features of different semantic levelsin the pixel domain.

Considering that the dependencies among different levelsof features can be estimated from both local to global viewand global to local view; we utilize the bidirectional GRU tomodel the relations from two different views. The Bi-GRUcontains the forward GRU

−→f which reads from v1 to v4, and

backward GRU←−f which reads from v4 to v1:−→lt =

−−−→GRU(vt), t ∈ [1, 4] (5)

←−lt =

←−−−GRU(vt), t ∈ [1, 4] (6)

For each timestep t, we obtain the hidden representations byconcatenating the forward hidden state

−→lt and the backward

hidden state←−lt , i.e., lt = [

−→lt ,←−lt ], which compose the final

semantic feature representation L = {lt}, t ∈ [1, 4].

D. Fusion Sub-network

We assume that the physical and semantic features ofimages are complementary in detecting fake news; thus wepropose the fusion sub-network to fuse these features. Specif-ically, we utilize the output vector of the frequency domainsub-network l0 and pixel domain sub-network {l1, l2, l3, l4} tomake predictions. {l1, l2, l3, l4} represent visual features fromdifferent semantic levels while l0 represents visual featurein the physical level. Intuitively, not all features contributeequally to the task of fake news detection, which means somevisual features play a more important role than others inevaluating whether a given image is a fake-news or real-newsimage. For instance, for some tampered images which presentsobvious tampering traces, the physical features perform betterthan semantic features; for some misleading images whichhave not experienced severe re-compressions, the semanticfeatures are more effective. Therefore, we highlight thosevaluable features via attention mechanism, and the enhancedimage representation is computed as follows:

F(li) = vT tanh(Wf li + bf ), i ∈ [0, 4] (7)

αi =exp (F (li))∑i exp (F (li))

(8)

u =∑i

αili (9)

where Wf denotes the weight matrix and bf is the biasterm, vT denotes a transposed weight vector and F is thescore function which measures the significance of each featurevector. Afterward, we obtain the normalized weight of i-thfeature vector αi via a softmax function and compute theadvanced representation of the input image as a weightedsum of different feature vectors. The vector v is randomlyinitialized and jointly learned during the training process.

Till now, we have obtained a high-level representation uof the input image, which models the characteristics of thisimage at both physical and semantic levels. We use a fullyconnected layer with softmax activation to project this vectorinto the target space of two classes: fake-news image and real-news image, and gain the probability distributions:

p = softmax (Wcu+ bc) (10)

In the proposed model MVNN, we define the loss function ascross-entropy error between the predicted probability distribu-tion and the ground truth:

L = −∑

[y log p+ (1− y) log (1− p)] (11)

where y is the ground truth with 1 representing fake-newsimage and 0 representing real-news image and p denotes thepredicted probability of being fake-news image.

V. EXPERIMENTS

In this section, we conduct experiments on a real-worlddataset to evaluate the effectiveness of the proposed modelMVNN. Specifically, we aim to answer the following evalua-tion questions:

• EQ1: Is MVNN able to improve the performance of fakenews detection based on visual modality?

• EQ2: How effective are different domains and other net-work components: attention, Bi-GRU and branches in thepixel domain sub-network, in improving the performanceof MVNN?

• EQ3: Can MVNN help improve the performance ofmultimodal fake news detection?

We first introduce the dataset we used, and some representativemethods which models the visual contents for fake news detec-tion as baselines; next we describe the implementation detailsof the proposed MVNN model. And then we compare MVNNwith those baselines to answer EQ1, and we investigate theablation study to answer EQ2, which includes quantitativeand qualitative analyses. To answer EQ3, we further compareMVNN with baselines on multimodal fake news detection.Last, we study some typical cases to vividly illustrate theimportance of multiple domains for fake news detection.

A. Dataset

Considering that fake news detection based on multimediacontent is a pretty new task, there are a few standard mul-timedia fake news datasets available. The two most widelyused datasets are Twitter dataset presented in the MediaEvalVerifying Multimedia Use benchmark [7], and Weibo datasetbuilt in [16]. However, there are a lot of duplicate images in theTwitter dataset, causing the amount of distinctive images lessthan 500, which makes Twitter dataset too small to support thetraining of the proposed model. Therefore, in this paper, weconduct our experiments on Weibo dataset alone to evaluatethe effectiveness of the proposed model.

In Weibo dataset, the fake news are crawled from May, 2012to January, 2016 and verified by the official rumor debunkingsystem of Weibo1, which actually serves as an reputable sourceto collect fake news posts in literature. The real news arecollected during the same period as the fake news from Weiboand are verified by Xinhua News Agency, an authoritativenews agency in China. According to [16], duplicated and verysmall images have been removed for ensuring the quality ofentire dataset. To ensure that each post corresponds to an

1https://service.account.weibo.com/

image, text-only posts are removed and only one image issaved for posts with multiple illustrations. In total, this datasetincludes 4749 fake news posts and 4779 real news posts withcorresponding images. In our experiment, we first use K-means algorithm to cluster all news posts into 200 clusters,and split the whole dataset into training set, validation set andtesting set to ensure that there is no event overlap betweenthese sets, which further prevents the model from overfitting onevent topics. The training, validation and testing sets containa number of posts approximately with a ratio of 7:1:2 as in[17].

B. Baselines

To validate the effectiveness of MVNN, we choose severalrepresentative methods which are used to model the visualcontents for fake news detection as baselines:

• Forensics Features (FF)+LR: Reference [15] proposesnew image forensics features and evaluate the effective-ness of these features in detecting fake news. We adoptlogistic regression (LR) algorithm to take use of thesefeatures to make classifications.

• Pre-trained VGG: Pre-trained VGG are widely used asa feature extractor in existing studies about multimodalfake news detection [16]–[18]. Specifically, these worksuse the output of the last layer of a 19-layer VGGNetpre-trained on ImageNet, of which the feature dimensionis 4096. Similarly, we use LR to classify these features.

• Fine-tuned VGG: It has become a common practice thatfine tuning a pre-trained model on a task-relevant datasetfor achieving a promising performance on specific tasks.Hence, in addition to Pre-trained VGG, we also use Fine-tuned VGG as a compared experiment. In particular, weuse the training data to fine tune pre-trained VGG19,and use the same method as Pre-trained VGG to extractfeatures and make classifications.

• ConvAE: AutoEncoder (AE) is a kind of artificial neuralnetwork used to learn efficient data codings in an unsu-pervised manner. Convolutional AutoEncoder (ConvAE)[35] extends the AE framework through utilizing convo-lutional layer to compose the encoder and decoder forobtaining better understanding of images than plain fullyconnected layers. Considering that only a few methodsare used to model the visual contents for fake newsdetection, we use ConvAE as an additional comparisonmethod to show the full advantage of the proposed model.In order to avoid any bias, we design the encoder tobe the same structure as the pixel domain sub-networkwithout branches. After pre-training the ConvAE, weuse the encoder to extract features and use LR to makeclassifications.

C. Implementation Details

In this section, we introduce the implementation details ofthe proposed model for reproducibility, including parametersetting and training strategy. In the frequency domain sub-network, the two fully connected layers contain 16 and 64

TABLE I: The performance comparison of fake news detectionin single visual modality.

Method Accuracy Precision Recall F1FF+LR 0.650 0.612 0.579 0.595Pre-trained VGG 0.721 0.669 0.738 0.702Fine-tuned VGG 0.754 0.74 0.689 0.714ConvAE 0.734 0.685 0.744 0.713MVNN 0.846 0.809 0.857 0.832

neurons respectively, and we add a dropout layer with arate of 0.4 after the last fully connected layer to avoidoverfitting. In the pixel domain sub-network, the number ofhidden units of the GRU and fully connected layer is 32 and64, and each fully connected layer is followed by a dropoutlayer with a rate of 0.5. The number of branches is also ahyperparameter, and we find that 4 performs best after a lotof experiments. The frequency and pixel domain sub-networksare pre-trained and further fine tuned during the jointly trainingprocess. When pre-training the pixel domain sub-network, weuse data augmentation strategy to improve its generalizationperformance. In the whole network, we add a lot of batchnormalization layers to accelerate its convergence, and we setthe batch size to be 64. The model is trained for 300 epochswith early stopping to prevent overfitting. We use ReLU asthe non-linear activation function, and use Adam algorithm tooptimize the loss function.

D. Performance Comparison

In this subsection, we conduct some experiments to comparethe performance of MVNN and existing baselines described inSection V-B (EQ1). We use the accuracy, precision, recall andF1 score of the fake-news image class as evaluation metrics.

From Tab. I, we can see that: 1) MVNN is much betterthan other baselines, which validates that MVNN can effec-tively capture the intrinsic characteristics of fake-news images.Specifically, MVNN achieves an accuracy of 84.6%, outper-forming existing approaches by 9.2% at least. 2) Intuitively,Fine-tuned VGG performs better than Pre-trained VGG, indi-cating that the learned features are more relevant to the taskof fake news detection after fine-tuning the model on the fakenews dataset. 3) The performance of ConvAE is slightly betterthan Pre-trained VGG. This demonstrates that ConvAE has theability of understanding universal semantics of images, whichis similar to models pre-trained in a supervised manner. 4) Theperformance of FF+LR is the worst among these comparedmethods, because the information captured by these forensicsfeatures is very limited.

E. Ablation Study

In this subsection, we aim to evaluate the effectiveness ofmultiple domains and other network components in improvingthe performance of MVNN (EQ2) from the quantitative andqualitative perspectives.Quantitative Analysis

TABLE II: Ablation study of MVNN.

Method Accuracy Precision Recall F1MVNN 0.846 0.809 0.857 0.832w/o frequency domain 0.794 0.792 0.728 0.758w/o pixel domain 0.737 0.698 0.717 0.708w/o attention 0.827 0.778 0.853 0.814w/o Bi-GRU 0.828 0.772 0.841 0.805w/o branches 0.803 0.752 0.830 0.789

To intuitively illustrate the effectiveness of different do-mains and other network components, we design several in-ternal models for comparison, which are simplified variationsof MVNN by removing certain components:

• w/o frequency domain (fd): The frequency domain sub-network is removed from MVNN while the attentionmechanism is saved for fusing features of the pixeldomain sub-network.

• w/o pixel domain (pd): We removed the pixel domainsub-network and associated attention mechanism. Theremaining structure is the frequency domain sub-networkand the binary classifier.

• w/o attention: MVNN without the attention mechanism.Features from the frequency and pixel domain sub-networks are concatenated to make classification.

• w/o Bi-GRU: MVNN without the Bi-GRU in the pixeldomain sub-network.

• w/o branches: In addition to remove the Bi-GRU, wealso remove the branches in the pixel domain sub-network. The input image is fed into the pixel domainsub-network and obtains an output vector from the lastblock, which is further used to make classification.

The results of the ablation study are reported in Tab. II. Wehave the following observations:

1) Multiple domains: The frequency and pixel domain bothplay important roles for fake news detection, especially thepixel domain. Specifically, the accuracy drops by 5.2% and10.9% without the frequency and pixel domain sub-networkcorrespondingly, which further demonstrates that physical andsemantic visual information are both important for detect-ing fake news. Besides, the performance of MVNN withoutpixel domain is obviously worse than frequency domain. Itillustrates that pixel domain plays a major role in fake newsdetection while frequency domain is auxiliary.

2) Network Components: Other than multiple domains,other network components: attention, Bi-GRU and branchesin pixel domain sub-network, are all important for achievingthe best performance by MVNN. If we remove one or severalof them, the performance would drop by a certain degree.Specifically, i) the accuracy are 1.9% lower than the wholemodel if we remove attention, which means that the attentionmechanism can fuse physical and semantic visual featurevectors better than simply concatenating, further proving ourassumption that these features contribute to the task dynami-cally; ii) When we remove the Bi-GRU from the pixel domainsub-network, the performance of MVNN reduces 1.8%; when

(a) (b) (c)

Fig. 7: Visualization of learned latent visual feature representations on the testing data. Dots in red and blue represent fake-newsimages and real-news images, respectively. (a) Frequency domain sub-network. (b) Pixel domain sub-network. (c) MVNN.

we further remove the branches, the performance drops by4.3%. Therefore, we can conclude that incorporating differentlevels of features and considering the dependencies betweenthese features both help capture the semantic characteristicsof visual contents.Qualitative Analysis

In the previous ablation experiments, we have intuitivelyevaluated the utility of multiple domains in improving theperformance of MVNN. In this subsection, we qualitativelyvisualize the visual features learned by the frequency domainsub-network, the pixel domain sub-network and MVNN whichconsiders both two domains on the testing set with t-SNE [36],as shown in Fig. 7. The label for each image is real-newsimage or fake-news image.

From Fig. 7, we can easily observe that the separabilityof the feature representations learned by these three networkscan be sorted as: MVNN > the pixel domain sub-network> the frequency domain sub-network. Specifically, in thevisualization graph of the frequency domain sub-network,the feature representations of positive and negative samplesoverlap a lot. This is because all images uploaded to socialmedia platforms will be compressed, which covers the originalcompression or tampering traces to some extent, reducing thedifference between fake-news images and real-news imagesin frequency domain. For the pixel domain sub-network,it can learn discriminable features, but the learned featuresare still twisted together, especially in the middle part ofFig. 7b. In comparison, in the visualization graph of MVNN,these is a relatively visible boundary between samples withdifferent labels. From above phenomena, we can concludethat: i) pixel domain is more effective than frequency domainin distinguishing fake-news images and real-news images;and ii) frequency and pixel domains are complementary fordetecting fake-news images, consequently MVNN which fusesthe information of multiple domains can learn better and moredistinctive feature representations, and thus achieves betterperformance than single domain sub-network.

F. Application on Multimodal Fake News Detection

In the previous parts, we have proven the effectiveness ofMVNN in detecting fake news utilizing visual contents alone.Considering that a complete news story consists of text and

visual content, latest researches usually utilize multimodalinformation to detect fake news. Following the convention, wecompare the performance of different visual representationsobtained from above visual modeling methods in fake newsdetection under the condition of fusing text information (EQ3),as described in Tab. III. We experiment with three state-of-the-art fusing methods as follows:

• attRNN: Reference [16] proposes an innovative RNNwith an attention mechanism for effectively fusing thetextual, visual and social context features. In detail, theneuron attention from the output of the LSTM is utilizedwhen fusing with the visual features. In our experiments,we remove the part dealing with social context features.

• EANN: Reference [17] designs an event adversarial neu-ral network to learn event-invariant multimodal featuresthrough adversarial training techniques. It consists of amultimodal feature extractor, a fake news detector andan event discriminator.

• MVAE: Reference [18] utilizes a multimodal variationalautoencoder trained jointly with a fake news detectorto learn a shared representation of textual and visualinformation for fake news detection. MVAE is composedof an encoder, a decoder and a fake news detector.

Tab. III shows the similar trend as Tab. I. From Tab. III, weobserve that MVNN consistently outperforms other baselines

TABLE III: The performance comparison of fake news detec-tion in multi-modality.

Method Accuracy Precision Recall F1

attRNN

FF+LR 0.735 0.801 0.665 0.727Pre-trained VGG 0.821 0.813 0.862 0.837Fine-tuned VGG 0.849 0.888 0.818 0.852ConvAE 0.816 0.848 0.796 0.821MVNN 0.901 0.911 0.901 0.906

EANN

FF+LR 0.780 0.840 0.724 0.778Pre-trained VGG 0.821 0.861 0.791 0.824Fine-tuned VGG 0.841 0.883 0.807 0.843ConvAE 0.823 0.863 0.794 0.827MVNN 0.897 0.930 0.872 0.900

MVAE

FF+LR 0.777 0.776 0.815 0.795Pre-trained VGG 0.813 0.893 0.737 0.804Fine-tuned VGG 0.832 0.875 0.798 0.835ConvAE 0.827 0.831 0.847 0.839MVNN 0.891 0.896 0.898 0.897

0.98

0.11

0.92Frequency

Pixel

MVNN 0.69

0.44

0.78Frequency

Pixel

MVNN

(a)

0.98

0.11

0.92Frequency

Pixel

MVNN 0.69

0.44

0.78Frequency

Pixel

MVNN

(b)

Fig. 8: Some fake-news images detected by the frequencydomain sub-network and MVNN but missed by the pixeldomain sub-network.

by a large margin on all fusion methods. Specifically, MVNNexceeds compared methods by over 5.2% in accuracy, whichsuggests that MVNN can easily replace existing methods toobtain the representations of visual contents, for achievingsignificant improvements in detecting fake news. Moreover,the performance of FF+LR in attRNN is obviously worse thanEANN and MVAE, because attRNN can hardly utilize thesemantic alignment between the text and the forensics featuresto fuse the textual and visual information.

G. Case Studies

In order to further illustrate the importance of multiple do-mains for fake news detection, we compare the results reportedby MVNN and single domain sub-networks, and exhibit somesuccessful examples which are captured by MVNN but missedby single domain sub-networks.

Fig. 8 shows two fake-news images detected by the fre-quency domain sub-network and MVNN but missed by thepixel domain sub-network. Fig. 8a was used to make up a fakenews post about a lost child, and Fig. 8b is an illustration of afake news which says that juice drinks can cause leukemia inchildren. The image in the upper left is the original imageattached in the fake news post, and the histogram in theupper right shows the variance of DCT coefficients on 64frequencies, which reflects the characteristics of this imagein the frequency domain. In general, the larger the varianceis, the heavier the image has been re-compressed. From theperspective of semantics, the two images in Fig. 8 do notshow evidence of fake news while their frequency histogramslook quite suspicious, showing that they are very likely to beoutdated images. The scores obtained from MVNN and singledomain sub-networks are listed in the bottom of each figure.We can observe that these two examples are correctly classifiedby MVNN while the results would totally change without thefrequency information, which further proves the importance ofthe frequency domain information in fake news detection.

Fig. 9 shows another two fake-news images detected bythe pixel domain sub-network and MVNN but missed by thefrequency domain sub-network. Fig. 9a tells a fake news thatthe South Korean President Lee Myung-bak knelt down to

0.83

0.80

0.36Frequency

Pixel

MVNN 0.82

0.93

0.42Frequency

Pixel

MVNN

(a)

0.83

0.80

0.36Frequency

Pixel

MVNN 0.82

0.93

0.42Frequency

Pixel

MVNN

(b)

Fig. 9: Some fake-news images detected by the pixel domainsub-network and MVNN but missed by the frequency domainsub-network.

apologize to the nation for the sexual assault of a seven-year-old girl. In fact, he was just praying. Fig. 9b is used tofabricate a fake news story about a huge snake which is justa mock-up. We can see that the two frequency histograms inFig. 9 show little evidence of fake news, however, the imagecontents are eye-attracting and rather dubious. By combiningthe information of the frequency and pixel domain, MVNNcan easily detect that this is fake-news image with highconfidence. Therefore, the information of the pixel domainis also important for detecting fake news.

VI. CONCLUSIONS AND FUTURE WORK

In this paper, we propose a novel framework MVNN tomodel the visual contents for fake news detection, which takesadvantage of the visual information of frequency and pixeldomains to effectively capture and fuse the characteristics offake-news images at physical and semantic levels. Experimentsconducted on Weibo dataset validate the effectiveness ofMVNN, and further prove the importance of multiple domainsin fake news detection.

There are several future works that need further investi-gations. First, MVNN is a general framework for extractingeffective visual representations to detect fake news, which isnot limited to platforms. In this paper, we only evaluate theproposed model on Weibo data due to the limited distinctiveimages in the existing Twitter multimedia dataset. In futurework, we can construct a larger multimedia dataset fromTwitter platform and explore the generalization capacity of theproposed model on different datasets. Further, we can comparethe similarities and differences between the visual contents ofWeibo and Twitter data. Second, although there are alreadymany studies focusing on fusing multimodal information forfake news detection, it is still a challenging problem whichneeds further investigation. For example, we can use the se-mantic alignment between images and text to explore the roleof different modalities. At last, how to explain the decisionsmade by existing models based on multimodal information isworth considering, since it can help us further understand anddefend fake news.

ACKNOWLEDGMENT

REFERENCES

[1] H. Allcott and M. Gentzkow, “Social media and fake news in the 2016election,” Journal of Economic Perspectives, vol. 31, no. 2, pp. 211–36,2017.

[2] K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, “Fake news detection onsocial media: A data mining perspective,” ACM SIGKDD ExplorationsNewsletter, vol. 19, no. 1, pp. 22–36, 2017.

[3] S. Kumar and N. Shah, “False information on web and social media: Asurvey,” arXiv preprint arXiv:1804.08559, 2018.

[4] A. Zubiaga, A. Aker, K. Bontcheva, M. Liakata, and R. Procter,“Detection and resolution of rumours in social media: A survey,” ACMComputing Surveys (CSUR), vol. 51, no. 2, p. 32, 2018.

[5] L. Wu, F. Morstatter, X. Hu, and H. Liu, “Mining misinformation insocial media,” Big Data in Complex and Social Networks, pp. 123–152,2016.

[6] Z. Jin, J. Cao, Y. Zhang, J. Zhou, and Q. Tian, “Novel visual and statisti-cal image features for microblogs news verification,” IEEE Transactionson Multimedia, vol. 19, no. 3, pp. 598–608, 2017.

[7] C. Boididou, S. Papadopoulos, D.-T. Dang-Nguyen, G. Boato,M. Riegler, S. E. Middleton, A. Petlund, Y. Kompatsiaris et al.,“Verifying multimedia use at mediaeval 2016.” in MediaEval, 2016.

[8] S. Sun, H. Liu, J. He, and X. Du, “Detecting event rumors on sinaweibo automatically,” in Asia-Pacific Web Conference. Springer, 2013,pp. 120–131.

[9] C. Castillo, M. Mendoza, and B. Poblete, “Information credibility ontwitter,” in Proceedings of the 20th International Conference on WorldWide Web. ACM, 2011, pp. 675–684.

[10] K. Wu, S. Yang, and K. Q. Zhu, “False rumors detection on sina weiboby propagation structures,” in IEEE 31st International Conference onData Engineering (ICDE). IEEE, 2015, pp. 651–662.

[11] J. Ma, W. Gao, P. Mitra, S. Kwon, B. J. Jansen, K.-F. Wong, and M. Cha,“Detecting rumors from microblogs with recurrent neural networks.” inInternational Joint Conference on Artificial Intelligence, 2016, pp. 3818–3824.

[12] J. Ma, W. Gao, and K.-F. Wong, “Detect rumors on twitter by pro-moting information campaigns with generative adversarial learning,” inProceedings of the 2019 World Wide Web Conference. ACM, 2019.

[13] K. Wu, S. Yang, and K. Q. Zhu, “False rumors detection on sina weiboby propagation structures,” in 2015 IEEE 31st International Conferenceon Data Engineering. IEEE, 2015, pp. 651–662.

[14] K. Shu, S. Wang, and H. Liu, “Beyond news contents: The role ofsocial context for fake news detection,” Web Search and Data Mining,pp. 312–320, 2019.

[15] C. Boididou, S. Papadopoulos, D.-T. Dang-Nguyen, G. Boato, andY. Kompatsiaris, “The certh-unitn participation@ verifying multimediause 2015.” in MediaEval, 2015.

[16] Z. Jin, J. Cao, H. Guo, Y. Zhang, and J. Luo, “Multimodal fusionwith recurrent neural networks for rumor detection on microblogs,” inProceedings of the 2017 ACM on Multimedia Conference. ACM, 2017,pp. 795–816.

[17] Y. Wang, F. Ma, Z. Jin, Y. Yuan, G. Xun, K. Jha, L. Su, and J. Gao,“Eann: Event adversarial neural networks for multi-modal fake newsdetection,” in Proceedings of the 24th ACM SIGKDD InternationalConference on Knowledge Discovery & Data Mining. ACM, 2018,pp. 849–857.

[18] K. Dhruv, G. Jaipal Singh, G. Manish, and V. Vasudeva, “Mvae: Multi-modal variational autoencoder for fake news detection,” in Proceedingsof the 2019 World Wide Web Conference. ACM, 2019.

[19] K. Simonyan and A. Zisserman, “Very deep convolutional networks forlarge-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.

[20] Z. Jin, J. Cao, J. Luo, and Y. Zhang, “Image credibility analy-sis with effective domain transferred deep networks,” arXiv preprintarXiv:1611.05328, 2016.

[21] C. R. Sunstein, On Rumors. How Falsehoods Spread, Why We BelieveThem, What Can Be Done. Farrar, Straus and Giroux, 2009.

[22] P. J. Lang, “A bio-informational theory of emotional imagery,” Psy-chophysiology, vol. 16, no. 6, pp. 495–512, 1979.

[23] K. C. Dzmitry Bahdanau and Y. Bengio, “Neural machine translation byjointly learning to align and translate,” arXiv preprint arXiv:1409.0473,2014.

[24] F. Yang, Y. Liu, X. Yu, and M. Yang, “Automatic detection of rumor onsina weibo,” in Proceedings of the ACM SIGKDD Workshop on MiningData Semantics. ACM, 2012, p. 13.

[25] C. Boididou, K. Andreadou, S. Papadopoulos, D.-T. Dang-Nguyen,G. Boato, M. Riegler, Y. Kompatsiaris et al., “Verifying multimediause at mediaeval 2015.” in MediaEval, 2015.

[26] H. Allcott and M. Gentzkow, “Social media and fake news in the 2016election,” Journal of Economic Perspectives, vol. 31, no. 2, pp. 211–36,2017.

[27] Z. Lin, J. He, X. Tang, and C.-K. Tang, “Fast, automatic and fine-grained tampered jpeg image detection via dct coefficient analysis,”Pattern Recognition, vol. 42, no. 11, pp. 2492–2501, 2009.

[28] Y.-L. Chen and C.-T. Hsu, “Detecting recompression of jpeg images viaperiodicity analysis of compression artifacts for tampering detection,”IEEE Transactions on Information Forensics and Security, vol. 6, no. 2,pp. 396–406, 2011.

[29] T. Bianchi and A. Piva, “Image forgery localization via block-grainedanalysis of jpeg artifacts,” IEEE Transactions on Information Forensicsand Security, vol. 7, no. 3, pp. 1003–1017, 2012.

[30] Q. Wang and R. Zhang, “Double jpeg compression forensics basedon a convolutional neural network,” EURASIP Journal on InformationSecurity, vol. 2016, no. 1, p. 23, 2016.

[31] J. Yang, Y. Sun, J. Liang, Y.-L. Yang, and M.-M. Cheng, “Understandingimage impressiveness inspired by instantaneous human perceptual cues,”in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

[32] X. Zhu, L. Li, W. Zhang, T. Rao, M. Xu, Q. Huang, and D. Xu, “De-pendency exploitation: A unified cnn-rnn approach for visual emotionrecognition.” in International Joint Conference on Artificial Intelligence,2017, pp. 3595–3601.

[33] P. Zhang, D. Wang, H. Lu, H. Wang, and X. Ruan, “Amulet: Aggre-gating multi-level convolutional features for salient object detection,” inProceedings of the IEEE International Conference on Computer Vision,2017, pp. 202–211.

[34] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”in Proceedings of the IEEE Conference on Computer Vision and PatternRecognition, 2015, pp. 1–9.

[35] J. Masci, U. Meier, C. Dan, and J. Schmidhuber, “Stacked convolutionalauto-encoders for hierarchical feature extraction,” in International Con-ference on Artificial Neural Networks, 2011.

[36] L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journalof Machine Learning Research, vol. 9, no. Nov, pp. 2579–2605, 2008.