Slide 1

4th ACRE workshop, 21-23 Sep 2011, De Bilt

Observation Feedback from Reanalysis

Paul Poli, Cristian Codorean, Hans Hersbach, Dick Dee

Slide 1

Slide 2

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 2

Observation Feedback from ReanalysisWhat is this?

Applications

Facility: Observation Feedback Archive

Your wish-list?

Slide 3

What is“Observation Feedback from Reanalysis” ? (1/2)Reanalysis “reconstructs” the weather by using all

available observations, to optimally determine an “optimal trajectory” of the weather over a few hours (say 12 hours here: the trajectory is issued by a forecast model)

We do this with a forward integration: data assimilation:

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 3

4DVAR

Analysis

Time

p

Analysis Analysis Analysis

Observations, with error estimates

00UTC 12UTC 00UTC

An ancient date An ancient date + 1 day

Background forecast (propagates forward previous information, constrained by dynamical and physical relationships), with error estimates

Slide 4

What is “Observation Feedback from Reanalysis” ? (2/2)

In this integration, each observation is compared with the reanalysis product

- Before the observation is assimilated: Comparison with the background Difference is background departure,

also called observation innovation- After the observation is assimilated:

We compare with the analysis Difference is analysis departure,

also called observation residualAlso, the assimilation procedure may involve bias

correction, which means differences between observation and reanalysis should be considered

- Before bias correction: uncorrected background departure- After bias correction: (corrected) background departure4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 4

Observationwith error estimate

Background forecast

with error estimate

Analysis trajectory

Slide 5

All data in 20CR for year 1900

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 5

Break-down by report type:LAND / SHIP / Cyclone tracks

ISPD v2.2

Slide 6

Data used by 20CR for year 1900

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 6

ISPD v2.2

Break-down by reporting practice:SEA LEVEL / STATION LEVEL

Slide 7

Bias corrections applied by 20CR, year 1900

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 7

ISPD v2.2

Slide 8

Background departures (after bias correction) from 20CR, year 1900

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 8

ISPD v2.2

Slide 9

Background and analysis departures from 20CR, year 1900

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 9

ISPD v2.2

Slide 10

Also, learn from long time-series

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 10

• Data homogenization• U. Vienna

• For ERA-20C• ERA-CLIM (WP3) surface-pressure-only

reanalysis of the 20th century, using boundary and forcing data from (WP2)

• We hope to be able to learn from the 20CR feedback by running break detection algorithms developed by ERA-CLIM partners (WP4)

Slide 11

Generalization One wants to be able to look at

- Data counts

- But also

Mean,

Stdev,

RMS

Min, max ...

(aggregate functions)

- Of

Background departures

Uncorrected

Corrected

Analysis departures

And to break-down the results by:

- Dates and times

- Regional domains

- Altitude bands

Low areas

Mountains

…

- Observation report types

Surface stations

Ships

Buoys

…

- Usage types (used or not)

- Stations

- Any other dimension of interest4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 11

Slide 12

Letter to SantaDear Santa, Besides data counts by station and so on,I would like to be able to compare stations

in the same regions,compare data collections (looking for some

possible systematic errors…),compare different versions of databanks

(say ISPD v2.2 versus ISPD v3)compare different reanalyses,and slice all this in any dimension: map,

time-series…I have left milk and cookies for you by the

tree.

Seriously, is that asking too much?4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 12

Slide 13

Having (recently?) learnt that Santa *may* not exist (apologies if I disappoint anyone else…)We have chosen to follow a two-step approach:

First, build an infrastructure where all the needed information can be stored and retrieved:

- Observation Feedback Archive

Second, we have developed some ideas on how to build queries in a way that reduces the problem of computing a set of statistics into a unique SQL query

ERA-CLIM will deliver the first of these two pointsI will only talk briefly about the second point

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 13

Slide 14

Schematic contents of the Observation Feedback Archive Observation record attributes:

- Time and geolocation (lat, lon, alt.)- Observation report type (buoy…)- Geophysical variable (T,p,q…)- Source (=which databank),

collection, station name- Unique identifier*- Reporting practice*- Observation value- Feedback added value:

Model land-sea mask Model orography Background & analysis dep. Obs. bias correction estimate

(~ accuracy) Probable obs. error stdev.

(~ 1-sigma precision)4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 14

ObservationFeedbackArchive

Stations/ships(De Bilt, Lusitania…)

Data collections (Byrd Antarctic expeditions…)

Databanks (ISPD 2.2…)

Reanalyses (20CR, ERA-40…)

Observation record81513 Pa at station level (1861m) AGUASCAL, Mexico, 17 Sep 1900 at 23UTC

Supports SQL queries

Slide 15

Observation Feedback Archive Interface Browser

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 15

Reanalysis

Vertical sounding?

Assimilated?

Decade / Year / Month

Observation report type

Geophysical variable

Note: we will eventually display proper names instead of numbers (these numbers are only intelligible by persons familiar with the specifics of our observation archive nomenclature)

Q: SHOULD OTHER DIMENSIONS BE INCLUDED FOR

QUICK BROWSING?

Auto-update of the availability, based on selection

Slide 16

Anticipated functions of the interfaceLocate datasets rapidly

- In a few clicks on one pageSubmit a query

- Standard record attributes are returned to the user (text format)

Possible additions:User choice of the attributes to be retrievedBasic plotting of the spatio-temporal data coverage

- Map- Time-series of data counts

Other formats for returning data to users

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 16

Q: ANYTHING ELSE?

Slide 17

Second step…Assuming all the data are in a SQL-type database, how can you

get statistics computed for you? (without asking someone to write code to do this)

1. What is the domain of the search?- WHERE

2. What quantities are to be computed?- SELECT + aggregate functions, e.g. avg()

3. What are the independent coordinates against which you want to compute your statistics?- GROUP BY: can act on bins [], or discrete values

That’s all you need

A generic algorithm has been written to apply this logic to build a SQL query and decode results into a structured dataset (JSON)

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 17

Slide 18

Conclusions: Observation Feedback from Reanalysis• Observations + feedback from reanalyses will be stored in a

centralized repository, the Observation Feedback Archive• SQL functionality, back-up, etc…

• Will provide user-friendly…• …browsing of the data (via a web interface)• … access to the data (serving text files)

• Reanalysis feedback will provide powerful information on:• Consistency between various data sets• Systematic errors• Changes in behavior (breaks)

… in short … data quality and problems that remain to be solved!

• All outputs from our future reanalyses will be stored on this Observation Feedback Archive, a PUBLIC facility

• CONSEQUENTLY, WE WILL NOT USE OBSERVATIONS THAT HAVE RESTRICTIONS

4th ACRE workshop, 21-23 Sep 2011, De Bilt Slide 18