1

Why your work is important.

The critique of models and why sensitivity analysis comes in …

… models under enemy fire?

2

The critique of models (see book)

3

Today’s critique of models

4

Michael Crichton presents ‘adversarial’ opinion on retreating glaciers and thickness of Antarctic ice cap – sea levels are not rising, he contends.

5

“They talk as if simulation were real-world data. They ‘re not. That ‘s a problem that has to be fixed. I favor a stamp: WARNING: COMPUTER SIMULATION – MAY BE ERRONEOUS and UNVERIFIABLE. Like on cigarettes […]”, p. 556

6

For sure modelling is subject toady to an unprecedented critique, which is no longer limited to post-modern philosophers but involves intellectuals and scientists of different political hues.

Have models fallen out of grace and is modelling -- just useless arithmetic as claimed by Pilkey and Pilkey-Jarvis 2007?

7

Useless Arithmetic: Why Environmental Scientists Can't Predict the Futureby Orrin H. Pilkey and Linda Pilkey-Jarvis

Quantitative mathematical models used by policy makes and government administrators to form environmental policies are seriously flawed

8

One of the examples discussed concerns the Yucca Mountain repository for radioactive waste disposal, where a very large model called TSPA (for total system performance assessment) is used to guarantee the safe containment of the waste. TSPA is Composed of 286 sub-models.

9

TSPA (like any other model) relies on assumptions -- a crucial one being the low permeability of the geological formation and hence the long time needed for the water to percolate from the desert surface to the level of the underground disposal.

The confidence of the stakeholders in TSPA was not helped when evidence was produced which could lead to an upward revision of 4 orders of magnitude of this parameter.

10

We just can’t predict, concludes N. N. Taleb, and we are victims of the ludic fallacy, of delusion of uncertainty, and so on. Modelling is just another attempt to ‘Platonify’ reality…

Nassim Nichola Taleb, The Black Swan,

Penguin, London 2007

11

Many will disagree with the works just cited. Yet our point is the cat is out of the bag already. Stakeholders and media alike will tend to expect or suspect instrumental use of computation models, amplification or dampening of uncertainty as a function of convenience and so on.

Note for the students:The book has a good state of the art on the sea level rise story.

12

The critique of models <-> Uncertainty

Peter Kennedy, A Guide to Econometrics

Anticipating criticism by applying sensitivity analysis. This is one of the ten commandments of applied econometrics according to Peter Kennedy:

“Thou shall confess in the presence of sensitivity.

Corollary: Thou shall anticipate criticism ’’

13

The spectre of type III errors: “Assessing the wrong problem by incorrectly accepting the false meta-hypothesis that there is no difference between the boundaries of a problem, as defined by the analyst, and the actual boundaries of the problem” (Dunn, 1997). = answering the wrong question = framing issue (Peter Kennedy’s 2nd commandment of applied econometrics: ‘Thou shall answer the right question’, Kennedy 2007)

Dunn, W. N.: 1997, Cognitive Impairment and Social Problem Solving: Some Tests for Type III Errors in Policy Analysis, Graduate School of Public and International Affairs, University of Pittsburgh, Pittsburgh.

14

In sensitivity analysis: Type I error: assessing as important a non important factor

Type II: assessing as non important an important factor

Type III: analysing the wrong problem

15

Type III in sensitivity: Example:•In the case of TSPA (Yucca mountain) a range of 0.02 to 1 millimetre per year was used for percolation of flux rate. Applying sensitivity analysis to TSPA could or could not identify this as a crucial factor, but this would be of scarce use if the value of the percolation flux were later found to be of the order of 3,000 millimetres per year.