data discovery and access to the international surface pressure databank (ispd)
DESCRIPTION
Data Discovery and Access to The International Surface Pressure Databank (ISPD). Thomas Cram Gilbert P. Compo* Doug Schuster Chesley McColl* Steven Worley National Center for Atmospheric Research, Boulder, CO *NOAA/CIRES, Boulder, CO. AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2. - PowerPoint PPT PresentationTRANSCRIPT
1
Data Discovery and Access to The International Surface Pressure
Databank (ISPD)
Thomas Cram Gilbert P. Compo*Doug Schuster Chesley McColl*Steven Worley
National Center for Atmospheric Research, Boulder, CO*NOAA/CIRES, Boulder, CO
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
2
Research Data Archive (RDA) at NCARrda.ucar.edu
1. 600+ distinct datasets for climate and weather research
2. Collections: ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs
3. Free and open accesshttp://rda.ucar.edu
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
3
ISPD Overview• World’s largest collection of surface & sea level pressure observations
Land station Marine observations (ICOADS) Tropical cyclone best track
(IBTrACS)• Period (version 2): 1768 – 2010• Volume: 465 Gbyte• Available since Aug 2010
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
4
ISPD Overview• 60+ Contributors
o Atmospheric Circulation Reconstructions over the Earth (ACRE)o Australian Bureau of Meteorologyo British Antarctic Surveyo Cook Islands Meteorological Serviceo Danish Meteorological Instituteo Deutscher Wetterdienst (DWD; German Weather Service)o European and North Atlantic Daily to Multidecadal Climate Variability
(EMULATE)o ETH Zurich, Switzerlando GCOS/WCRP Working Group on Observational Data Sets for
Reanalysiso MANY MANY MORE….
• Assembled by NOAA/ESRL, CIRES (Univ. of Colorado), & NOAA/NCDC
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
5
ISPD Sample Annual Station Distribution
1850 * Land stations only* No marine stations
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
6
18501900
* Land stations only* No marine stations
ISPD Sample Annual Station Distribution
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
7
18501900
1950
* Land stations only* No marine stations
ISPD Sample Annual Station Distribution
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
8
18501900
19502000
* Land stations only* No marine stations
ISPD Sample Annual Station Distribution
9
ISPD Observations/Year
Figure courtesy Chesley McColl, NOAA/ESRL
2010: 53 Million
~ 1.5 Billion total observations
10
Data Access: Problem Background• Large computational/storage resources
needed– Store data– Extract desired data from large grids/files– Convert data to desirable format(s)
Scientific data centers have these resources
Individual researchers generally don’t
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
11
• Goals– Make data more accessible and easier to use for
individual researchers• Reasonable access volumes• Desired data formats• User defined parameters/grids
• Researchers stay focused on research
Data Access: Problem Background
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
12
ISPD Data Access Services
• Powerful computing resources @ NCAR
• Large disk storage (~ 0.5 PB)• Rich and detailed metadata• Direct file download via web• Customized data sub-setting• HDF-5 to ASCII software tools
(NetCDF forthcoming)
13
ISPD Metadata Features• Both group- and file-level metadata• Drive interfaces for file grouping and sub-
setting tools• Support efficient back-end processing• Improve scalability• Provide “quick look” at data samples
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
14
ISPD Metadata Interface Example
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
15AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
16AGU 2012 Fall Meeting: IN44A-05
17AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
18
Data Access: ISPD Subset Interface
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
19
ISPD Data Access ServicesTemporal range sub-
setting (daily)
Spatial sub-setting Lat/Lon region Individual station ID
20
ISPD Data Access Services
Data sub-setting options (cont.)• Observation type
Land station Marine obs Radiosonde Dropsonde TC best track
21
ISPD Data Access Services
• Subsetting processed in delayed mode• E-mail notification• Download via server-provided scripts
(wget)
22
ISPD 2012 Subset metrics
Data accessed: ~ 6.5 TB
Data served: ~ 46 GB
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
23
Summary & Future Directions
• RDA – Supply “User Friendly” Data Parameter & spatial sub-setting Metadata discovery Format conversion Improved and additional services
• NWSC-Cheyenne opening – more computing power
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
24
• DOI assignment
• Geoscience Data Journal article
• ISPD v3 (1755-2010) Spring 2013
ISPD Forthcoming
AMS 2013 Annual Meeting: 29EIPT/25CVC J1.2
