nowcasting of earthquake consequences using big social data · assessment of natural disasters from...

9
Big Data 2 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING Nowcasting of Earthquake Consequences Using Big Social Data Marco Avvenuti Department of Information Engineering, University of Pisa Stefano Cresci and Mariantonietta N. La Polla Institute of Informatics and Telematics, IIT-CNR, Italy Carlo Meletti National Institute of Geophysics and Volcanology, Italy Maurizio Tesconi Institute of Informatics and Telematics, IIT-CNR, Italy Messages posted to social media in the aftermath of a natural disaster have value beyond detecting the event itself. Mining such deliberately dropped digital traces allows a precise situational awareness, to help provide a timely estimate of the disaster’s consequences on the population and infrastructures. Yet, to date, the automatic assessment of damage has received little attention. Here, the authors explore feeding predictive models by tweets conveying on-the- ground social sensors’ observations, to nowcast the perceived intensity of earthquakes. T he large user base, interactive nature, and ubiquity of mobile social media platforms have made them primary hubs for public expres- sion and interaction. The unprecedented amount of situational observations con- veyed by social media users rose from the paradigm of social sensing, enabling new context-aware and predictive appli- cations. Within this context, social media users can be considered social sensors, namely “humans as citizens on the ubiquitous Web, acting as sen- sors and sharing their observations and views using mobile devices and Web 2.0 services.” 1 In the paradigm of social sensing, humans are the sensors them- selves, as opposed to only being sensor carriers and operators. Emergency management is a prom- ising application domain for social sensing. 2 Yet, little effort has been devoted to obtain quick estimations of events’ consequences on the population and infrastructures. 3 The importance of crowdsourced social data, such as eye- witness reports, toward the estimation of damage have long been asserted. 4 However, final results of current sys- tems based on citizen reports might take days after the disaster’s occur- rence. Meanwhile, the citizen-sensed stream of on-the-ground observations risks remaining unheeded. Predictive models have long been trained on web and social media data to explain both real- and virtual-world phenomena (for others’ work in this

Upload: others

Post on 22-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Big

Dat

a

2 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING

Nowcasting of Earthquake Consequences Using Big Social Data

Marco AvvenutiDepartment of Information Engineering, University of Pisa

Stefano Cresci and Mariantonietta N. La PollaInstitute of Informatics and Telematics, IIT-CNR, Italy

Carlo MelettiNational Institute of Geophysics and Volcanology, Italy

Maurizio TesconiInstitute of Informatics and Telematics, IIT-CNR, Italy

Messages posted to social media in the aftermath of a natural disaster have

value beyond detecting the event itself. Mining such deliberately dropped digital

traces allows a precise situational awareness, to help provide a timely estimate

of the disaster’s consequences on the population and infrastructures. Yet, to

date, the automatic assessment of damage has received little attention. Here,

the authors explore feeding predictive models by tweets conveying on-the-

ground social sensors’ observations, to nowcast the perceived intensity of

earthquakes.

T he large user base, interactive nature, and ubiquity of mobile social media platforms have made

them primary hubs for public expres-sion and interaction. The unprecedented amount of situational observations con-veyed by social media users rose from the paradigm of social sensing, enabling new context-aware and predictive appli-cations. Within this context, social media users can be considered social sensors, namely “humans as citizens on the ubiquitous Web, acting as sen-sors and sharing their observations and views using mobile devices and Web 2.0 services.”1 In the paradigm of social sensing, humans are the sensors them-selves, as opposed to only being sensor carriers and operators.

Emergency management is a prom-ising application domain for social sensing.2 Yet, little effort has been devoted to obtain quick estimations of events’ consequences on the population and infrastructures.3 The importance of crowdsourced social data, such as eye-witness reports, toward the estimation of damage have long been asserted.4 However, final results of current sys-tems based on citizen reports might take days after the disaster’s occur-rence. Meanwhile, the citizen-sensed stream of on-the-ground observations risks remaining unheeded.

Predictive models have long been trained on web and social media data to explain both real- and virtual-world phenomena (for others’ work in this

Page 2: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Nowcasting of Earthquake Consequences Using Big Social Data

SEPTEMBER/OCTOBER 2017 3

area, see the related sidebar). Results have been obtained in many fields, either successful, as with syndromic surveillance5 and citations,6 or controversial, as with political elections.7 To date, little effort has been put into predic-tive models capable of defining (in an accurate

and timely manner) a disaster’s severity per-ceived by eyewitnesses.8 In an effort to deter-mine whether we can leverage big social data in a responsive system that’s able to nowcast disaster consequences, we evaluated the abil-ity of a set of predictive linear models to map

Related Work in Social Media Analysis for Disaster Management

Recently, a large body of work was devoted to automati-cally detecting emergencies by analyzing the anoma-

lies in social media communication patterns.1 In this work, emergency-related keywords are sought in the real-time message stream and anomaly/burst detection algorithms are applied by analyzing word frequencies in fixed-width time windows. Comparisons with statistical baselines allow the identification of new emergencies. Other researchers have achieved important results for the detection of earth-quakes,2–4 and more recently for wildfires,5 traffic jams,6 and transport breakdowns.7

Other works have focused on making sense of emergency-related messages. Given the sheer number of messages shared during disasters, researchers have developed means to auto-matically identify the most relevant information contained among large sets of messages.8–10 Such works largely adopt natural language processing techniques, combined with pow-erful machine learning algorithms, to obtain a limited number of highly relevant messages manually analyzed or further pro-cessed by other systems.

In an effort to obtain a clearer picture of unfolding disas-ters, academia has also proposed means to produce crisis maps from social media data. Typically, crisis mapping systems produce a geographic map of an area struck by a disaster and different parts of the map are colored according to the emer-gency’s severity, as inferred from social media messages11–13 Furthermore, such maps can be complemented with additional information directly extracted from relevant messages, such as eyewitness reports of damage or multimedia content. Other systems aren’t focused specifically on crisis maps, but instead provide a wide set of statistics and interactive visualizations via Web interfaces.2,14,15

Finally, one particular study16 presents a survey on compu-tational techniques for social media data processing in emer-gencies; this can be considered for further references in this field.

References 1. M. Avvenuti et al., “A Framework for Detecting Unfolding Emergencies

Using Humans as Sensors,” SpringerPlus, vol. 5, no. 1, 2016, pp. 1–23.

2. M. Avvenuti et al., “EARS (Earthquake Alert and Report System): A Real

Time Decision Support System for Earthquake Crisis Management,” Proc.

20th ACM SIGKDD Int’l Conf. Knowledge Discovery and Data Mining, 2014,

pp. 1749–1758.

3. P.S. Earle, D.C. Bowden, and M. Guy, “Twitter Earthquake Detection: Earth-

quake Monitoring in a Social World,” Annals of Geophysics, vol. 54, no. 6,

2012; http://dx.doi.org/10.4401/ag-5364.

4. T. Sakaki, M. Okazaki, and Y. Matsuo, “Tweet Analysis for Real-Time Event

Detection and Earthquake Reporting System Development,” IEEE Trans.

Knowledge and Data Eng., vol. 25, no. 4, 2013, pp. 919–931.

5. R. Power, B. Robinson, and D. Ratcliffe, “Finding Fires with Twitter,” Proc.

Australasian Language Technology Association Workshop, 2013, pp. 80–89.

6. E. D’Andrea et al., “Real-Time Detection of Traffic from Twitter Stream

Analysis,” IEEE Trans. Intelligent Transportation Systems, vol. 16, no. 4, 2015;

pp. 2269–2283.

7. M. Congosto, D. Fuentes-Lorenzo, and L. Sanchez, “Microbloggers as Sen-

sors for Public Transport Breakdowns,” IEEE Internet Computing, vol. 19, no. 6,

2015, pp. 18–25.

8. S. Cresci et al., “A Linguistically-Driven Approach to Cross-Event Damage

Assessment of Natural Disasters from Social Media Messages,” Proc. 24th

Int’l Conf. World Wide Web Companion, 2015, pp. 1195–1200.

9. M. Imran et al., “Extracting Information Nuggets from Disaster-Related

Messages in Social Media,” Proc. 10th Int’l Information Systems for Crisis

Response and Management Conf., 2013, pp. 791–801.

10. S. Verma et al., “Natural Language Processing to the Rescue? Extracting

‘Situational Awareness’ Tweets During Mass Emergency,” Proc. 5th Int’l

AAAI Conf. Web and Social Media, 2011; www.aaai.org/ocs/index.php/ICWSM

/ICWSM11/paper/view/2834.

11. S. Cresci et al., “Crisis Mapping during Natural Disasters via Text Analysis

of Social Media Messages,” Web Information Systems Engineering, Springer,

2015, pp. 250–258.

12. R. Goolsby. “Social Media as Crisis Platform: The Future of Community

Maps/Crisis Maps,” ACM Trans. Intelligent Systems and Technology, vol. 1, no. 1,

2010, article no. 7.

13. S.E. Middleton, L. Middleton, and S. Modafferi, “Real-Time Crisis Mapping of

Natural Disasters Using Social Media,” IEEE Intelligent Systems, vol. 29, no. 2,

2014, pp. 9–17.

14. H. Purohit and A.P. Sheth, “Twitris v3: From Citizen Sensing to Analysis,

Coordination and Action,” Proc. 7th Int’l AAAI Conf. Web and Social Media,

2013; www.aaai.org/ocs/index.php/ICWSM/ICWSM13/paper/view/6106.

15. J. Yin et al., “Using Social Media to Enhance Emergency Situation Aware-

ness,” IEEE Intelligent Systems, vol. 27, no. 6, 2012, pp. 52–59.

16. M. Imran et al., “Processing Social Media Messages in Mass Emergency: A

Survey,” ACM Computing Surveys, vol. 47, no. 4, 2015, article no. 67.

Page 3: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Big Data

4 www.computer.org/internet/ IEEE INTERNET COMPUTING

the intensity of worldwide earthquakes. Here, we demonstrate that situational awareness per-ceived by social sensors and shared through Twitter can be exploited to predict the outcome of traditional authoritative assessments with great accuracy.

Going Beyond Event DetectionTo date, the majority of efforts for social media-based crisis management exploited bursts of messages to perform event detection. Figure 1a shows an example of a typical burst of Twitter messages generated by an earthquake.

Although earthquake detection from social media is possible and effective,9,10 the use of ad hoc equipment such as seismographs allows more timely and accurate results, also provid-ing additional information, such as magnitude and hypocenter. However, seismic networks are incapable of quantifying damage caused by earthquakes.11

The severity of an earthquake is described by both magnitude and intensity. Magnitude char-acterizes earthquakes by the energy released at the hypocenter. Although earthquakes with a high magnitude are more likely to cause

(a)

(b)

(c)

103.5

102.5

Freq

uenc

y (t

wee

ts/m

inut

e)

Time (GMT�1)

101.5

18:00 18:30 19:00 19:30 20:00 20:30 21:00 22:0021:30

103

102

Region

North America

Central and South America

Rest of the world

R2

0.4888

0.7160

0.5227

R2adj

0.4655

0.6640

0.4913

R2pred

0.3874

0.5597

0.4635

MAE

0.65

0.41

0.53

RMSE

0.82

0.56

0.78

n

734

182

147

p

32

28

9

p-value

�10�20

�10�20

�10�18

Longest tweet streak

Location hashtag count

STR � MOD unigrams count

Country hashtag count

Mean tweet distance from epicenter

Mean time delay between tweets

Variance tweet distance from epicenter

Minimum account distance from epicenter

Minimum tweet distance from epicenter

Mean account distance from epicenter

Time

Tweet

Linguistic

Tweet

Tweet

Time

Tweet

User

Tweet

User

4.7704

3.0336

1.5145

0.7744

�0.6728

�0.6022

0.5765

�0.5441

�0.4418

�0.1353

1

2

3

4

5

6

7

8

9

10

V27

V5

V45

V4

V12

V22

V14

V20

V13

V19

Rank Predictive variable Class Predictive power (b coef�cient)

Figure 1. Gathering and analyzing relevant social media data after an earthquake. (a) A trend of tweets containing the keyword “earthquake” shared before and after the 4.5 magnitude earthquake that struck near Edmond, Oklahoma in the US on 7 December 2013. (b) Earthquake intensity estimation results. (c) The top 10 predictive variables ranked by their global absolute predictive power. (MAE 5 mean absolute error; MOD 5 unigrams associated with moderate earthquakes; RMSE 5 root mean squared error; and STR 5 unigrams associated with strong earthquakes.)

Page 4: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Nowcasting of Earthquake Consequences Using Big Social Data

SEPTEMBER/OCTOBER 2017 5

damage, earthquakes’ consequences depend on many other factors, such as depth and dis-tance of the hypocenter, soil characteristics, and buildings’ vulnerability. Thus, the magni-tude can’t be considered a direct measure of the degree of damage, and another dimension — the intensity — is used to indicate the local effects of an earthquake. An estimation of the inten-sity can be derived from instrumental measure-ments; however, this approach doesn’t take into account information coming from the earth-quake-stricken area, and this could be improved greatly by analyzing data collected onsite.

The conviction that social media can be use-ful sources of information is increasing among emergency stakeholders.3,12 In fact, data collected from web surveys is already employed to assess earthquake damage. The “Did You Feel It?” (DYFI) tool12 used by the US Geological Survey (USGS) collects and analyzes experiences and obser-vations of registered citizens by inviting them to take a simple web survey (http://earthquake .usgs.gov/dyfi) whenever an earthquake occurs in the vicinity of their hometown. Questions are designed so that citizen responses can be auto-matically translated into earthquake intensity values by a simple algorithm. The DYFI system outputs intensity estimations on a 1 to 10 scale, with the minimum value indicating an earth-quake with no perceivable effects, and the maxi-mum value indicating a devastating earthquake.

Despite this being an interesting approach to the exploitation of crowdsourced knowledge, the DYFI system lacks responsiveness, with the final outcome being released hours/days after the disaster has occurred. On the other hand, a timely estimation of severe earthquakes would allow responders to put in place a more effective response, thus prioritizing resources and efforts whenever and wherever they’re most needed.13 For these reasons, the study, design, and devel-opment of techniques able to extract knowledge from big social data in a timely manner is a cru-cial research challenge. Here, our aim is to pro-vide a timely estimation of the DYFI earthquake intensity values from tweets. To achieve our goal, we trained a set of ordinary-least-squares predictive models, which exploit linear correla-tions between the predictive variables and the quantity to be predicted, namely USGS’s offi-cial DYFI intensity value. USGS’s DYFI values serve as an authoritative ground truth, while tweets are exploited to compute our predictive

variables. This task is intrinsically hard, given the heterogeneity between subjective tweets, often produced by scared eyewitnesses, and authoritative intensity values deriving from objective analyses by domain experts.

Social Signs of Earthquake IntensityThe first step toward nowcasting earthquake intensity from social data consists of defin-ing the link between earthquakes and tweets. For each earthquake detected by the USGS, we selected only those tweets shared during a given time window after the earthquake’s occurrence. Among selected tweets, we retained only those tweets written in the most widely spoken lan-guage in the country where the earthquake occurred. For instance, we selected English tweets for earthquakes occurring in the US, and Spanish tweets for earthquakes occurring in Puerto Rico, Mexico, Chile, and so on. Then, we computed 45 numeric variables that serve as potential predictors of earthquake intensity.

Predictive variables are built upon the data made available by Twitter and fall into four dif-ferent classes, according to the nature of the information they aim to capture. The first class of variables exploits the structure of tweets and their metadata. Specifically, we found hashtags to be particularly useful because eyewitness reports usually carry #earthquake or #quake hashtags (similarly, Spanish messages carry #temblor, #terremoto, or #choque). Regarding the structure of tweets, our previous findings highlighted that the emotional state of users after an emergency is reflected in the length, use of punctuation, and number of capital let-ters in the messages shared.9 Indeed, the more scared users are, the more they tend to share shorter messages, with a less-complex and less-defined structure. These characteristics can possibly be used as predictors of the extent of damage. Furthermore, correlations between the spatial distribution of tweets around the epicen-ter and the earthquake intensity were assessed whenever geographic (GPS) data was available. Thus, we defined the following set of variables: the total number of tweets (V1); ratio between the total number of tweets and the mean num-ber of tweets shared during the same time of day for all other days (V2); number of #earth-quake (or similar) hashtags (V3); number of hashtags with the name of the country involved (V4); number of hashtags with the name of the

Page 5: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Big Data

6 www.computer.org/internet/ IEEE INTERNET COMPUTING

location hit by the earthquake (V5); mean (V6) and variance (V7) of the total number of words among messages; mean (V8) and variance (V9) of the total number of characters among messages; mean ratio between the number of capital let-ters and total number of characters in messages (V10); mean ratio between the number of punc-tuation characters and total number of charac-ters in messages (V11); and mean (V12), minimum (V13), and variance (V14) of tweets’ distances from the epicenter.

The second class of predictive variables, built upon user metadata such as a user’s home account location, can help us understand whether a relation exists between earthquake intensity and the spatial distribution of users reporting the earthquake. We exploited the Geonames gaz-etteer (www.geonames.org) for the conversion from the user location string to the geographic coordinates. The resulting seven variables are as follows: the number of distinct accounts (V15); number of distinct accounts that tweeted from the same country of the earthquake (V16); number of distinct accounts that tweeted from a neigh-boring country (V17); number of distinct coun-tries derived from the accounts’ locations (V18); and mean (V19), minimum (V20), and variance (V21) of accounts’ distances from the epicenter.

The third class of variables leverages the publication timestamp to quantify the time distribution of tweets, and aims at grasping the bursty nature of emergency communica-tions.8 Other previous studies have exploited bursty characteristics of message streams for the tasks of topic or event detection, and here we want to evaluate whether the quantifica-tion of such characteristics contributes to the estimation of the intensity of earthquakes. We therefore designed the following set of predic-tive variables: the mean time delay between one message and the next one (V22); mean (V23), minimum (V24), maximum (V25), and variance (V26) in the number of messages per minute; and longest streak of messages having a maximum delay of 5 seconds between one another (V27).

The fourth class of variables is derived from linguistic features of tweets. Variables of this class count specific keywords among tweets. Our choice of keywords is automatic, and it exploits a modified version of a proto-words detection algorithm.14 By providing the algo-rithm with a set of seed events (such as high-intensity earthquakes), it’s possible to extract

prototypical expressions representative of mes-sages related to such earthquakes. Then we can exploit these prototypical expressions to com-pute our linguistic variables. Given |C| classes of earthquakes, each class ci ∈ C is represented by a set of seed earthquakes Si 5 {ei,1, …, ei,n}. We denote as Ti 5 {ti,1, …, ti,m} the set of tweets associated to the seed earthquakes of class ci. Then, we compute frequencies for every uni-gram hk found in a tweet t ∈ Ti:

τ τ τ η= ∈ ∧ηT T{ : contains }i i k, k

η ηc

T

Tseedfreq ( , ) = .k i

i

i

, k

The term seedfreq(hk, ci) represents the nor-malized frequency of the unigram hk for the class ci. The normalization term |Ti| refers to all the tweets (including those, for example, that don’t contain the unigram hk) associated to the seed earthquakes ei, and is necessary to account for the different cardinalities among such sets of tweets. The score of the unigram hk for the class ci is then computed as

∑η

η

η=

cc

c

score ( , ) =seedfreq( , )

seedfreq( , )

.k ik i

k jj

C

1

This score indicates how much a unigram hk is representative of the class ci. For our experi-ments, we chose three classes of earthquakes: strong earthquakes that caused damage and casualties (STR); moderate earthquakes widely felt by the population, but without severe con-sequences (MOD); and light earthquakes felt only by a few people (LIG). We picked the top 10 uni-grams from the STR class as predictors, together with the top five unigrams from the MOD class. Unigrams of the LIG class aren’t directly exploited to compute variables, but instead serve as contrast terms to highlight the typical expressions of the other classes. This resulted in 10 variables computed as the number of times a unigram of the STR class was used in the earth-quake reports (V28, …, V37), plus five variables computed the same way for unigrams of the MOD class (V38, …, V42). We also added aggre-gate variables: the total number of unigrams of the STR class (V43); total number of unigrams of the MOD class (V44); and total number of uni-grams from both the STR and MOD classes (V45).

Page 6: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Nowcasting of Earthquake Consequences Using Big Social Data

SEPTEMBER/OCTOBER 2017 7

Before running the proto-words detection algo-rithm, message texts were preprocessed with the Python NLTK framework (www.nltk.org) by applying normalization, stopwords removal, tokenization, and stemming.

To build the dataset for our study, we started collecting earthquake-related tweets in English and Spanish for a period of 90 days, spanning from 18 October 2013 to 15 January 2014. To get real-time access to the global stream of newly produced tweets, we used Twitter’s Streaming API. Relevant tweets were selected by keywords commonly adopted in earthquake-related tweets and already proposed in the literature, such as “earthquake” and “shaking” for English, and “choque,” “temblor,” and “terremoto” for Span-ish.9,10 We then queried USGS for DYFI intensity values of worldwide earthquakes that occurred during the same time window. We finally built a dataset of up to 5 million tweets related to 7,283 globally distributed earthquakes.

Quick Estimates from Social SignsWe modeled DYFI earthquake intensity values as a linear combination of our 45 predictive variables, plus terms for pairwise interactions:

∑β β γ ε= + + +=

y V I .i j j i i ij

0 ,1

45

In the definition of our model, yi represents the intensity of the ith earthquake; b0 is the intercept term of our linear model; bj are the coefficients of our variables Vj,i; g is the coef-ficient vector of the interaction terms; Ii is the vector of the pairwise interactions between variables, and ei represents the error term.

To train the predictive models, we first divided the 7,283 earthquakes into three groups: first, earthquakes that occurred in the US and Canada (the “North America” group); second, earthquakes that occurred in Puerto Rico, Mex-ico, Chile, Peru, Argentina, and so forth (the “Central and South America” group); and third, the remaining earthquakes belonging to the “Rest of the world” group. Then, for each group, we trained a predictive model that aims to esti-mate the DYFI value of the earthquakes of that group. Figure 1b shows the evaluation results for the three models we trained.

The proposed models have been evaluated by means of R2, Adjusted R R( )adj

2 2 , Predicted R R( )pred

2 2 , as well as mean absolute error (MAE)

and root mean squared error (RMSE). The Rpred

2 metric is a form of leave-one-out cross-validation. It’s particularly suitable to assess the goodness-of-fit of a model with regards to unseen observations, and is useful to control the risk of overfitted models. Indeed, it’s pos-sible to avoid overfitting and assess a model’s ability to generalize by analyzing R2 and Radj

2 values versus Rpred

2 values. In contrast to R2 and Radj

2 , Rpred2 values drop as an overfitted

model loses its ability to generalize. Error val-ues for MAE and RMSE have to be considered in relation to the 1 → 10 DYFI intensity scale. For every model, we also report the number of observations n and the number of predictors p included in the model. All the models pro-posed in Figure 1b show p-values ,, 0.001 assessing their statistical significance.

Overall, our models are able to estimate earthquake intensity with a percentage MAE error of 5.3 percent, and the best-performing model shows a percentage MAE error as low as 4.1 percent. Estimations for earthquakes of the North American region show increased errors: MAE 0.65 versus 0.41 and 0.53, RMSE 0.82 versus 0.56 and 0.78. This is because 98.5 per-cent of the earthquakes of the North American region had a magnitude value between 2 and 4, while earthquakes in the two other regions had magnitude values almost always higher than 4.

Figure 1b also shows that estimations for earthquakes occurring within the Central and South American region present lower errors than those occurring in the rest of the world. This is mainly due to constraining our analyses to only tweets in English and Spanish. The per-formance reduction for such earthquakes can be estimated in 22–28 percent less-accurate pre-dictions: MAE 0.53 versus 0.41 and RMSE 0.78 versus 0.56. It’s worth noting, however, that despite a 0.19 reduction in R2, the “Rest of the world” model exhibits a MAE value of 0.53 that still reflects accurate predictions.

Figure 1c reports the 10 variables account-ing for the highest predictive power among all the trained models. This allows us to gain insights into which variables are more impor-tant for this task.

Furthermore, Figure 2a reports an analysis of the spatial distribution of prediction errors for earthquakes occurring within the US ter-ritory. Given that the majority of earthquakes occurred in regions of high seismic hazard (such

Page 7: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Big Data

8 www.computer.org/internet/ IEEE INTERNET COMPUTING

as California), we employed a hexagonal bin-ning technique to avoid overplotting and obtain a more readable map. Groups of earthquakes are represented by hexagons of area proportional to the number of earthquakes in the group, while the color represents the RMSE of earthquake inten-sity predictions for each group. As we show, few high prediction errors are associated with smaller hexagons, typically representing a single seismic event, while in areas affected by a large number

of earthquakes our predictions are overall accu-rate. Furthermore, the few orange-colored hexa-gons are spread throughout the map, where no regions with errors considerably higher or lower than the average exist. In turn, this indicates the lack of a geographic bias within our model.

Besides the small error in intensity esti-mates, another promising result of the study is the responsiveness of our approach. The aver-age delay of our estimations is in the order of

(a)

Root mean squared error (RMSE) between real and predicted intensity Number of earthquakes

0.0 1.0 2.0 3.0

(b)

Figure 2. Earthquake distribution and reports. (a) Hexagonal binning distribution of intensity estimation errors for earthquakes occurring in the US. Each hexagon represents the root mean squared error (RMSE) of the earthquakes occurred in a limited geographic area. Wider hexagons represent areas where a larger number of earthquakes have occurred. Green hexagons mean low RMSE values (∼− 0), while red hexagons mean RMSE values ∼− 3. RMSE values have to be considered in relation to the 1 → 10 earthquake intensity scale. (b) Examples of earthquake reports related to the 6.0 magnitude earthquake that occurred in the South Napa region of California on 24 August 2014.

Page 8: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Nowcasting of Earthquake Consequences Using Big Social Data

SEPTEMBER/OCTOBER 2017 9

100 minutes, which could be further reduced by shortening the time window used to col-lect earthquake reports from the Twitter stream when the accuracy of the prediction can be traded off for responsiveness.

Back to SocietyThese promising results seem to confirm the possibility to estimate the damage produced by earthquakes via accurate analyses of social signs extracted from user reports, thus allowing responders to rapidly identify potentially severe earthquakes.

The fluctuations of accurately estimat-ing intensity between our models highlight an important difference between social mining systems and systems based on seismographs. The latter aren’t influenced by the earthquake’s magnitude and also provide accurate analy-ses in the case of light tremors. Being based on spontaneous user reports, our approach is affected by the lack of social sensors in sparsely populated areas, or by the lack of messages in the case of light tremors. However, this aspect shouldn’t raise much concern, because light seismic events don’t pose a serious threat to communities and infrastructures, while the earthquakes of interest are those actually felt by the population at large.

Variables from all four classes appear among those yielding the highest contribution to our intensity estimations. The first three variables, namely V27, V5, and V45, provide a contribution that’s significantly higher than the remaining ones, thus representing the most important pre-dictors of earthquake intensity.

Notably, 5 out of 10 variables, namely V12, V14, V20, V13, and V19, are based on the account’s location field or GPS geolocation associated to tweets. This further stresses the role of geo-graphic information as a key contributing factor for this task, and demonstrates the cor-relation existing between the geographic distri-bution of reports and the intensity. We consider these results to be even more interesting, con-sidering that we didn’t apply any geolocation technique to the analyzed tweets. That is, we only exploited natively (GPS) geolocated tweets to compute such variables. Thus, the predictive power of geographic variables, and the accuracy of the resulting models, could further increase by employing geoparsing techniques that let us augment the number of geolocated tweets.13

Furthermore, an analysis of the messages shared after severe earthquakes highlighted that many tweets contain reports and photos of specific places/buildings that suffered dam-age (see Figure 2b).15 Automated image and text analysis techniques could be employed to fur-ther analyze tweets in the aftermath of high-intensity earthquakes, thus allowing responders to collect timely, specific mentions of damage. Such information could then be organized and shown via a web interface. In fact, given the loose requirements of our approach, a fully working system for social media-based intensity estimation could be implemented by exploiting the analysis pipelines of already existing web emergency management systems.3,9

G iven this picture, directions for future work are manifold and include the possibility of

carrying this approach over to other natural disasters such as hurricanes, flash floods, land-slides, or even man-made offenses such as ter-rorist attacks or financial crises. Many of these emergencies lack the timely and detailed charac-terization that’s available for earthquakes thanks to seismographic networks. On the one hand, this lack of quantitative descriptive data could repre-sent an added difficulty to the training of dam-age models and, ultimately, to the deployment of such predictive damage assessment systems. On the other hand, the lack of ad hoc sensing equipment renders other sources of informa-tion, such as social media, even more valuable. Other directions of improvement might consist in using the proposed approach to rapidly iden-tify and contact users who are directly involved in the emergency, for instance eyewitness users, with the purpose of acquiring a better situ-ational awareness. For example, users tweeting from a scarcely covered region could be directly asked to provide more detailed information.

As a final remark, we could think of the broad research field of social sensing, where emergency management represents just one among the possible applications, as a means to foster civic involvement and improve social good. Processing spontaneous user reports and feeding results back to society could in fact initiate virtuous circles between communities, researchers, and emergency stakeholders, pav-ing the way for the development of more sus-tainable and resilient societies.

Page 9: Nowcasting of Earthquake Consequences Using Big Social Data · Assessment of Natural Disasters from Social Media Messages,” Proc. 24th Int’l Conf. World Wide Web Companion, 2015,

Big Data

10 www.computer.org/internet/ IEEE INTERNET COMPUTING

AcknowledgmentsThis research was partially supported by the EU H2020 Pro-

gram under the scheme INFRAIA-1-2014-2015: Research

Infrastructures grant agreement 654024 SoBigData: Social

Mining & Big Data Ecosystem, and by the PRA 2016 project

“Analysis of sensory data: from traditional sensors to social

sensors” funded by the University of Pisa.

References 1. A.P. Sheth, “Citizen Sensing, Social Signals, and

Enriching Human Experience,” IEEE Internet Comput-

ing, vol. 13, no. 4, 2009, pp. 87–92.

2. H. Gao, G. Barbier, and R. Goolsby, “Harnessing the

Crowdsourcing Power of Social Media for Disaster

Relief,” IEEE Intelligent Systems, 2011, pp. 10–14.

3. J. Yin et al., “Using Social Media to Enhance Emer-

gency Situation Awareness,” IEEE Intelligent Systems,

vol. 27, no. 6, 2012, pp. 52–59.

4. M. Avvenuti et al., “Predictability or Early Warning:

Using Social Media in Modern Emergency Response,”

IEEE Internet Computing, vol. 20, no. 6, 2016,

pp. 4–6.

5. J. Ginsberg et al., “Detecting Influenza Epidemics

Using Search Engine Query Data,” Nature, vol. 457,

no. 7232, 2009, pp. 1012–1014.

6. G. Eysenbach, “Can Tweets Predict Citations? Met-

rics of Social Impact Based on Twitter and Correlation

with Traditional Metrics of Scientific Impact,” J. Medi-

cal Internet Research, vol. 13, no. 4, 2011; doi: 10.2196

/jmir.2012.

7. D. Gayo-Avello. “No, You Cannot Predict Elections

with Twitter,” IEEE Internet Computing, vol. 16, no. 6,

2012, pp. 91–94.

8. P.S. Earle, D.C. Bowden, and M. Guy, “Twitter Earth-

quake Detection: Earthquake Monitoring in a Social

World,” Annals of Geophysics, vol. 54, no. 6, 2012;

http://dx.doi.org/10.4401/ag-5364.

9. M. Avvenuti et al., “EARS (Earthquake Alert and

Report System): A Real Time Decision Support System

for Earthquake Crisis Management,” Proc. 20th ACM

SIGKDD Int’l Conf. Knowledge Discovery and Data

Mining, 2014, pp. 1749–1758.

10. T. Sakaki, M. Okazaki, and Y. Matsuo, “Tweet Analysis

for Real-Time Event Detection and Earthquake Report-

ing System Development,” IEEE Trans. Knowledge and

Data Eng., vol. 25, no. 4, 2013, pp. 919–931.

11. S. Cresci et al., “A Linguistically-Driven Approach to

Cross-Event Damage Assessment of Natural Disasters

from Social Media Messages,” Proc. 24th Int’l Conf.

World Wide Web Companion, 2015, pp. 1195–1200.

12. P. Meier, “New Information Technologies and Their

Impact on the Humanitarian Sector,” Int’l Rev. Red

Cross, vol. 93, no. 844, 2011, pp. 1239–1263.

13. M. Avvenuti et al., “Impromptu Crisis Mapping to Pri-

oritize Emergency Response,” Computer, vol. 49, no. 5,

2016, pp. 28–37.

14. M. Pennacchiotti and A.-M. Popescu, “Democrats,

Republicans and Starbucks Afficionados: User Clas-

sification in Twitter,” Proc. 17th ACM SIGKDD Int’l

Conference on Knowledge Discovery and Data Mining,

2011, pp. 430–438.

15. M. Avvenuti et al., “Earthquake Emergency Management

by Social Sensing,” Proc. 12th IEEE Int’l Conf. Pervasive

Computing and Comm. Workshops, 2014, pp. 587–592.

Marco Avvenuti is a professor of operating systems in the

Department of Information Engineering at the Univer-

sity of Pisa. His research interests include human-cen-

tric sensing and social media analysis. Avvenuti has

a PhD in information engineering from the University

of Pisa. He’s a member of the IEEE Computer Society.

Contact him at [email protected].

Stefano Cresci is a PhD student in the Department of Infor-

mation Engineering at the University of Pisa and a

research fellow at IIT-CNR, Italy. His research interests

include social media mining and knowledge discovery.

Cresci has an MSc in computer engineering and an

MSc in big data analytics and social mining from the

University of Pisa. He’s a student member of IEEE and

member of the IEEE Computer Society. Contact him at

[email protected].

Mariantonietta N. La Polla is a postdoctoral researcher at

IIT-CNR in Pisa, Italy. Her research interests include

network analysis and its applications to social media

data, social media intelligence, and social sensing.

La Polla has a PhD in information engineering from

the University of Pisa. Contact her at mariantonietta

[email protected].

Carlo Meleti is a senior technological scientist at the

National Institute of Geophysics and Volcanology

(INGV), Italy. His research interests include seismic

hazard, seismic risk, and loss-reduction policy. Meletti

has an MSc in geological sciences from the University

of Pisa. Contact him at [email protected].

Maurizio Tesconi is a computer science researcher at IIT-

CNR. His research interests include social web mining,

social network analysis, and visual analytics within

the context of open source intelligence. Tesconi has a

PhD in information engineering from the University

of Pisa. He’s a member of the permanent team of the

European Laboratory on Big Data Analytics and Social

Mining. Contact him at [email protected].