what uncertainties do we need in bayesian deep learning ... · what uncertainties do we need in...

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Alex Kendall ([email protected]) and Yarin Gal ([email protected]) In Bayesian modelling, there are two main types of uncertainty we can model [1]: • Epistemic uncertainty: uncertainty in the model, capturing what our model doesn’t know due to lack of training data. Can be explained away with increased training data. • Aleatoric uncertainty: information which our data cannot explain. Can be explained away with increased sensor precision. It is important to model aleatoric uncertainty for: • Large data situations, where epistemic uncertainty is explained away, • Real-time applications, because we can form aleatoric models without expensive MC samples. • Noisy data, because we can learn to attenuate erroneous labels. with deep learning. Our model’s uncertainty for pixel output " is given by: " ≈ 1 ) , - / + 1 ) , - / − 1 ) , / - Using Monte Carlo dropout samples, T, learning aleatoric uncertainty with loss: = 1 ) 1 2 " " " − () " - + log " Input Image Ground Truth Model Segmentation Aleatoric Uncertainty Epistemic Uncertainty Experiments training on one dataset and testing on another. • Aleatoric uncertainty cannot be explained away with more data, • Aleatoric uncertainty does not increase for out-of-data examples (situations different from training set), • Epistemic uncertainty increases with decreasing training size, • Epistemic uncertainty increases with examples out of the training distribution. for semantic segmentation and per- pixel depth regression datasets. We use a convolutional network based on DenseNet [20] with 103 layers and 9.4M parameters Per-pixel depth regression Modelling uncertainty allows the model to learn to attenuate the effect from erroneous labels and learn loss attenuation. Input Image Depth Regression Uncertainty And epistemic uncertainty is important for: • Safety-critical applications, because epistemic uncertainty is required to understand examples which are different from training data, • Small datasets where the training data is sparse. 1. Types of Uncertainty 2. We jointly model aleatoric and epistemic uncertainty 3. SOTA performance 4. Uncertainty with Distance from Training Data 5. Conclusions

Upload: others

Post on 31-May-2020

6 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: What Uncertainties Do We Need in Bayesian Deep Learning ... · What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Alex Kendall (agk34@cam.ac.uk) and Yarin

WhatUncertaintiesDoWeNeedinBayesianDeepLearningforComputerVision?AlexKendall([email protected])andYarinGal([email protected])

InBayesianmodelling,therearetwomaintypesofuncertaintywecanmodel[1]:• Epistemicuncertainty:uncertaintyinthemodel,capturingwhatourmodeldoesn’tknowduetolackoftrainingdata.Canbeexplainedawaywithincreasedtrainingdata.

• Aleatoricuncertainty:informationwhichourdatacannotexplain.Canbeexplainedawaywithincreasedsensorprecision.

Itisimportanttomodelaleatoric uncertaintyfor:• Largedatasituations,whereepistemicuncertaintyisexplainedaway,• Real-timeapplications,becausewecanformaleatoricmodelswithoutexpensiveMCsamples.

• Noisydata,becausewecanlearntoattenuateerroneouslabels.

withdeeplearning.Ourmodel’suncertaintyforpixeloutput𝑦" isgivenby:

𝑉𝑎𝑟 𝑦" ≈1𝑇)𝜎 𝑥, -

�

+1𝑇)𝑓 𝑥, -

�

−1𝑇)𝑓 𝑥,

�

UsingMonteCarlodropoutsamples,T,learningaleatoricuncertaintywithloss:

𝐿𝑜𝑠𝑠 𝜃 =1𝐷)

12𝜎 𝑥 "

�

𝑦" − 𝑓(𝑥)"- + log𝜎 𝑥 "

InputImageGroundTruthModelSegmentationAleatoricUncertaintyEpistemicUncertainty

Experimentstrainingononedatasetandtestingonanother.• Aleatoricuncertaintycannotbeexplainedawaywithmoredata,

• Aleatoricuncertaintydoesnotincreaseforout-of-dataexamples(situationsdifferentfromtrainingset),

• Epistemicuncertaintyincreaseswithdecreasingtrainingsize,

• Epistemicuncertaintyincreaseswithexamplesoutofthetrainingdistribution.

forsemanticsegmentationandper-pixeldepthregressiondatasets.

WeuseaconvolutionalnetworkbasedonDenseNet [20]with103layersand9.4Mparameters

Per-pixeldepthregression

Modellinguncertaintyallowsthemodeltolearntoattenuatetheeffectfromerroneouslabelsandlearnlossattenuation.

InputImage DepthRegressionUncertainty

Andepistemic uncertaintyisimportantfor:• Safety-criticalapplications,becauseepistemicuncertaintyisrequiredtounderstandexampleswhicharedifferentfromtrainingdata,

• Smalldatasetswherethetrainingdataissparse.

1.TypesofUncertainty 2.Wejointlymodelaleatoric andepistemicuncertainty 3.SOTAperformance

4.UncertaintywithDistancefromTrainingData

5.Conclusions

Uncertainty and Uncertainties - NRC: Home PageE. Droguett and Ali Mosleh, “Bayesian methodology for model uncertainty using model Performance data,” Risk Analysis, 28, No. 5, 1457-1476,

agk34, vb292, rc10001 @cam.ac.uk arXiv:1511.02680v1 [cs.CV

Two calibration methods for modeling streamflow and ...€¦ · 2004was used in calibration and uncertainty analysis. The method has) a Bayesian framework and determines the uncertainties

Practical Statistics – part III Dealing with systematic uncertainties · 2018. 11. 16. · Practical Statistics – part III ... prescription to make confidence intervals, Bayesian

Ý ´ , >ºhb Ý Ø } P Ê · market data using Bayesian theorem. Generally, forecasts derived by only expert judgment include many uncertainties but Bayesian update can solve this

Lecture 10: Bayesian Deep Learning - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture10.pdf · A. Kendal, Y. Gal, What Uncertainties Do We Need in Bayesian Deep Learning for

Uncertainties propagation in metamodel-based …imechanica.org/files/uncertainties-draft-libre_0.pdf · Uncertainties propagation in metamodel-based probabilistic optimization of

Kline Uncertainties