Bayesian deep learning and uncertainty quantification applied to induced seismicity locations at theGroningen gas field in the Netherlands – What do we need for safe AI?Chen Gu∗, Youssef M. Marzouk, M. Nafi Toksoz, Massachusetts Instititute of Technology

SUMMARY

Recently, with the increase of dense seismic monitoring net-works and the resultant massive seismic data all over the world(e.g., Groningen gas reservoir), the traditional geophysical al-gorithms have faced the “big data” and high dimensionalityissues and become inefficient and costly. Deep learning, as acandidate to mitigate the “big data” and high dimensionalityproblems, has started to be applied to many geophysical prob-lems; however, previous deep learning studies seldom considerthe model uncertainty, which may seriously impair reservoirproduction estimates, ground motion predictions, and seismicearly warnings. Bayesian neural networks provide a practi-cal solution to solve the uncertainty quantification problems indeep learning, i.e., to make AI safe. In this paper, we con-struct a Bayesian convolutional neural network and implementa stochastic regularized technique – dropout – to quantify theuncertainty of seismic location. This method has been ap-plied to induced seismicities in the Groningen gas field in theNetherlands.

INTRODUCTION

Induced seismicity occurs in many oil and gas fields due tofluid injection and/or extraction all over the world, includingAmerica, Canada, China, Europe and the Middle East. Manycountries have employed dense seismic networks to monitorinduced seismicity and to obtain information about the seismicsource physics, reservoir structures and ground motions. Themassive seismic data make the traditional geophysical methodsinefficient and costly.

Historically, Groningen has low local seismicity; however, inrecent years, the gas production in the Groningen field hascaused an increase of induced seismicity. To monitor the in-duced seismicity at Groningen gas field, more than 70 bore-holes (each borehole has one 3-component accelerometer andfour 3-component geophones) and 17 surface accelerometerswere deployed to monitor induced seismicity (Figure 1(b)).Since 2009, the Royal Netherlands Meteorological Institute(KINM) has recorded more than 1000 earthquakes (M<5).These events are generally smaller than M 5; however, theseevents, occurring in the reservoirs, are very shallow with fo-cal depths less than about 4 km. As a result, in Groningen,where oil fields are close to populated areas, these inducedearthquakes could produce ground accelerations high enoughto cause damage to local structures.

To better estimate the seismic impacts on infrastructures andmitigate seismic hazards, Shell instrumented 17 local housesand continuously monitored house motions from 09/2017 to10/2018 (Figure 1(b)). Figure 1(b) shows the location of in-duced seismicity during the house monitoring period on a Gronin-

gen map based on the KNMI catalog. The location of the in-duced seismicity correlated well with pre-existing faults.

The induced seismicity and subsurface structure in Gronin-gen was intensively studied and well documented by many re-search groups. Spica et al. (2018) gave an overview of struc-tures from surface to reservoir depth in the Groningen gas field.van Thienen-Visser and Breunese (2015) published a detailedreport about the seismicity and the fault map in Groningenfield. Willacy et al. (2018, 2019) applied a full-waveform-based event location algorithm to the Groningen induced seis-micity. However, little research has applied AI methods to theGroningen induced seismicity. With the recent tremendous in-crease of seismic data in the Groningen gas field, we are able toapply deep learning methods to study the Groningen inducedseismicity.

Deep learning has been successfully used in many areas, e.g.,image classification, natural language processing, etc. (Le-Cun et al., 2015; Goodfellow et al., 2016). In recent years,more and more deep learning algorithms have been applied togeoscience problems (Beroza, 2018; Kong et al., 2018; Bergenet al., 2019). Fully connected neural networks have been usedto solve traditional location and tomography problems (Araya-Polo et al., 2018; DeVries et al., 2018; Gu et al., 2018). Manystudies have applied convolutional neural networks to automat-ically locate earthquakes without manual phase-arrival picking(Huang et al., 2018; Zhang et al., 2018; Kriegerowski et al.,2018). Deep convolutional neural networks have also beenused to obtain fast approximate simulation of seismic waves(Moseley et al., 2018). Richardson (2018) has proposed a newfull waveform inversion algorithm with recurrent neural net-works.

These previous deep learning studies seldom consider uncer-tainty quantification of weights of neural net, choice of ar-chitecture, choice of hidden layers, etc, which may result inserious problems (e.g., automatic driving). Bayesian neuralnetworks provide a way to understand uncertainties of deeplearning system and make AI safe (Ghahramani, 2016). Thecomplexity of deep neural networks makes it difficult to de-rive an analytical expression of the probability distribution ofparameters in Bayesian neural nets, as well as the probabilitydistribution of the prediction. Gal (2016); Gal and Ghahra-mani (2016a,b) have proposed practical tools – Bayesian deepneural nets with stochastic regularization techniques (SRTs) –to quantify uncertainties in deep learning. These tools can bescaled to big data and complex neural nets.

We implement a Bayesian convolutional neural network witha stochastic regularization technique (SRT), MC dropout, tolocate induced seismicity in the Groningen gas field and quan-tify the location uncertainties. The neural networks were firsttrained with synthetic waveform data and then applied to realseismic data.

Bayesian deep neural networks and uncertainty quantification applied to induced seismicity

(a)

(b)

Figure 1: (a) Location stations in the Groningen gas field.Green triangles denote borehole geophones, yellow trianglesdenote accelerometers, and white squares denote house moni-toring. (b) Location of induced seismicity of the giant gas fieldin the Netherlands from 09/2017 to 10/2018. The blue star de-notes the location of the largest 01/08/2018 M3.4 event in thecatalog.

THEORY – BAYESIAN NEURAL NETWORKS

The probability distribution of the prediction yyy∗ giving testinginputs xxx∗ with Bayesian neural networks can be represented as

p(yyy∗|xxx∗,XXX ,YYY ) =∫

p(yyy∗|xxx∗,ωωω)p(ωωω|XXX ,YYY )dωωω. (1)

where XXX and YYY are training inputs and outputs. ω is a generalrepresentation of random variables in deep neural networks,e.g., weights and bias (fully connected neural layers), kernels(convolutional neural networks), and gates (recurrent neuralnetworks), depending on neural network structures. p(ωωω|XXX ,YYY )is hard to derive analytically. An approximation variationaldistribution qθ (ωωω) is always used instead. qθ (ωωω) can be evalu-ated by minimizing the Kullback-Leibler (KL) divergence (Kull-back and Leibler, 1951)

KL(qθ (ωωω)||p(ωωω|XXX ,YYY )) =∫

qθ (ωωω logqθ (ωωω)

p(ωωω|XXX ,YYY ). (2)

Then, Equation 1 can be approximated as

p(yyy∗|xxx∗,XXX ,YYY )≈∫

p(yyy∗|xxx∗,ωωω)q∗θ(ωωω)dωωω =: q∗

θ(yyy∗|xxx∗). (3)

For complex deep neural networks, the approximated distri-bution qθ (ωωω) are always evaluated by stochastic regulariza-tion tools during training, e.g., dropout, multiplicative Gaus-sian noise, and dropConnect (Gal, 2016).

Given the approximated distribution qθ (ωωω), we can generateT realizations of model parameters, i.e., stochastic forwardpasses, according to the posterior model distribution{

ωωωi}

i=1...T∼ q∗

θ(ωωω). (4)

Assuming a Gaussian likelihood of the prediction yyy

p(yyy∗| fff ωωω (xxx∗)

)= N

(yyy∗; fff ωωω (xxx∗),τ−1III

), (5)

where τ is the precision of the measurement, the mean andvariance of the prediction of Bayesian neural networks can bewritten as (Gal, 2016)

E[yyy∗]≈ 1T

T∑t=1

yyy∗t (xxx∗), (6)

Var[yyy∗]≈ τ−1 1T

T∑t=1

yyy∗t (xxx∗)T yyy∗t (xxx

∗)−E[yyy∗]TE[yyy∗]. (7)

In this study, we construct a Bayesian convolutional neural net-work to map the 2D seismic wave field to seismic locations(Figure 2). To stochastically inject noise to model during train-ing, we added one dropout layer after the last convolutionallayer and before the fully connected layer. The output of thedropout layer is passed to the fully connected layer and outputa three-dimensional location vector yyy. A regression loss func-tion is used to measure the mean-square-error during training.

Bayesian deep neural networks and uncertainty quantification applied to induced seismicity

2

conv1

2

conv2

2

conv3

5

conv4

5

dropout

1

fc6+regression

Figure 2: Architecture of Bayesian convolutional neural network.

(a)

(b)

(c)

Figure 3: (a) Location of 8775 synthetic sources (red dots) andcross-sections of 3-D velocity model. (b) Location of 8775synthetic sources (red dots) and 64 stations (green triangles) ina map view. (c) Normalized seismic time series at 64 stations.

TRAINING OF THE BAYESIAN NEURAL NETWORK

To train the Bayesian neural network, we first generate syn-thetic waveforms at trial earthquake locations (39 ∗ 45 ∗ 5 =8775) (Figure 3(a)). Figure 3(b) shows the locations of 64 sta-tions with the synthetic earthquake locations in a map view.One example of a 2D wavefield due to one synthetic event atyyy = (245.6, 598.4, 3) km is shown in Figure 3(c). The 2Dwavefields, i.e., 2D images, are inputs of the network. The3-D location vectors yyy are outputs of the network. We use syn-thetic waveform fields observed at 64 stations and location (yyy)pairs to train our convolutional neural network.

During the training, the dropout layer at the end of convolu-tional layers and before the fully connected layer stochasticallyinjects noise at each iteration.

TESTING AND UNCERTAINTY QUANTIFICATION

We test the Bayesian neural network with noise-perturbed syn-thetic wave fields. The testing data are generated by adding10% Gaussian random noise to the “clean” synthetic wave-fields.

To quantify the uncertainty of testing outputs, we apply theMC dropout method by generating T = 1000 realizations ofthe Bayesian neural network for each testing input, i.e., stochas-tic forward passes. At each realization, the dropout layer isbetween the last convolutional layer and the fully connectedlayer and the dropout probability is 0.2. The posterior meanand variations of locations can be calculated by Equation 7.We show the posterior mean of locations of 1775 events atdepth of 3 km in a map view in Figure 4. The posterior stan-dard deviations in X, Y, and Z directions of 1775 events areshown in Figure 5(a), 5(b), and 5(c).

The location uncertainties are significantly affected by the sta-tion coverage. The synthetic sources away from the regionwith dense stations have larger bias and variance. Another in-teresting finding is the small variance in the Z direction. Thiswaveform-based neural network location method provides goodresolution in the Z direction. Accurate location in depth is veryimportant in reservoir characterization.

Bayesian deep neural networks and uncertainty quantification applied to induced seismicity

Figure 4: Posterior mean of the locations of 1775 syntheticsources (red dots) at depth of 3km. Black dots show the groundtruth locations. Blue lines link the posterior mean locationsand the ground truth.

CONCLUSIONS

Local earthquakes in Groningen are generally smaller than M5; however, these events, occurring in the reservoirs, are veryshallow with focal depths less than about 4 km. As a result, inGroningen, where oil fields are close to populated areas, theseinduced earthquakes could produce ground accelerations highenough to cause damage to local structures. Accurate and fastlocation and uncertainty quantification for big volumes of seis-mic data can help to better predict ground motions and gener-ate more reliable building codes to mitigate seismic hazards.

We apply the Bayesian convolutional neural network, with theMC dropout technique, to locate the induced seismicities inGroningen and quantify the location uncertainty. The Bayesiandeep neural network-based algorithms work efficiently to quan-tify model uncertainty in deep learning, e.g., weights in neuralnetworks. However, the uncertainty due to data distribution,noise (errors in labels and measurements, etc.), and modelstructures (architectures, hidden layers, activation functions,etc.) of deep neural networks still need to be further studied.

ACKNOWLEDGMENTS

This research was supported by Total, Shell Global, and MITERL.

(a)

(b)

(c)

Figure 5: Posterior standard deviation in X, Y, and Z direc-tions.

Bayesian deep neural networks and uncertainty quantification applied to induced seismicity

REFERENCES

Araya-Polo, M., J. Jennings, A. Adler, and T. Dahlke, 2018,Deep-learning tomography: The Leading Edge, 37, 58–66.

Bergen, K. J., P. A. Johnson, M. V. de Hoop, and G. C. Beroza,2019, Machine learning for data-driven discovery in solidearth geoscience: Science, 363.

Beroza, G. C., 2018, Machine learning improves forecasts ofaftershock locations.

DeVries, P. M., F. Viegas, M. Wattenberg, and B. J. Meade,2018, Deep learning of aftershock patterns following largeearthquakes: Nature, 560, 632.

Gal, Y., 2016, Uncertainty in deep learning: PhD thesis, PhDthesis, University of Cambridge.

Gal, Y., and Z. Ghahramani, 2016a, Dropout as a bayesianapproximation: Representing model uncertainty in deeplearning: international conference on machine learning,1050–1059.

——–, 2016b, A theoretically grounded application of dropoutin recurrent neural networks: Advances in neural informa-tion processing systems, 1019–1027.

Ghahramani, Z., 2016, A history of bayesian neural networks:Presented at the NIPS Workshop on Bayesian Deep Learn-ing.

Goodfellow, I., Y. Bengio, and A. Courville, 2016, Deep learn-ing: MIT press.

Gu, C., O. Buyukozturk, and M. Nafi Toksoz, 2018, Deeplearning applied to induced-seismicity location in thegroningen gas field in the netherlands, in SEG TechnicalProgram Expanded Abstracts 2018: Society of ExplorationGeophysicists, 3067–3071.

Huang, L., J. Li, H. Hao, and X. Li, 2018, Micro-seismic eventdetection and location in underground mines by using con-volutional neural networks (cnn) and deep learning: Tun-nelling and Underground Space Technology, 81, 265–276.

Kong, Q., D. T. Trugman, Z. E. Ross, M. J. Bianco, B. J.Meade, and P. Gerstoft, 2018, Machine learning in seismol-ogy: Turning data into insights: Seismological ResearchLetters, 90, 3–14.

Kriegerowski, M., G. M. Petersen, H. Vasyura-Bathke, and M.Ohrnberger, 2018, A deep convolutional neural network forlocalization of clustered earthquakes based on multistationfull waveforms: Seismological Research Letters.

Kullback, S., and R. A. Leibler, 1951, On information and suf-ficiency: The annals of mathematical statistics, 22, 79–86.

LeCun, Y., Y. Bengio, and G. Hinton, 2015, Deep learning:nature, 521, 436.

Moseley, B., A. Markham, and T. Nissen-Meyer, 2018, Fastapproximate simulation of seismic waves with deep learn-ing: arXiv preprint arXiv:1807.06873.

Richardson, A., 2018, Seismic full-waveform inversion us-ing deep learning tools and techniques: arXiv preprintarXiv:1801.07232.

Spica, Z. J., N. Nakata, X. Liu, X. Campman, Z. Tang, andG. C. Beroza, 2018, The ambient seismic field at groningengas field: An overview from the surface to reservoir depth:Seismological Research Letters, 89, 1450–1466.

van Thienen-Visser, K., and J. Breunese, 2015, Induced seis-micity of the groningen gas field: History and recent devel-

opments: The Leading Edge, 34, 664–671.Willacy, C., E. van Dedem, S. Minisini, J. Li, J. W. Blok-

land, I. Das, and A. Droujinine, 2018, Application of full-waveform event location and moment-tensor inversion forgroningen induced seismicity: The Leading Edge, 37, 92–99.

——–, 2019, Full-waveform event location and moment tensorinversion for induced seismicity: Geophysics, 84, KS39–KS57.

Zhang, X., C. Yuan, J. Zhang, S. Liu, Z. Chen, and W. Li,2018, Automatic microseismic detection and location viathe deep-convolutional neural network, in SEG TechnicalProgram Expanded Abstracts 2018: Society of ExplorationGeophysicists, 3057–3061.