amrc meteorological data distribution system part i: data sources and archive
DESCRIPTION
AMRC Meteorological Data Distribution System Part I: Data Sources and Archive. K. Elena Willmot, Matthew A. Lazzara, Linda M. Keller, Nicole M. Schroeder Antarctic Meteorological Research Center Space Science and Engineering Center University of Wisconsin-Madison. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
AMRC Meteorological Data Distribution System Part I:Data Sources and Archive
K. Elena Willmot, Matthew A. Lazzara, Linda M. Keller, Nicole M. Schroeder
Antarctic Meteorological Research CenterSpace Science and Engineering Center
University of Wisconsin-Madison
Introduction
o AMRC since 1992...o Part I Outline:
o Data SourcesInternal & External
o Data Formatso Data Archive
Historical & Present
o Distributiono Challenges
No more FTP?
Access and quality of data
Lack of acknowledgements
o Statistics
(Photo by Michael Forster Rothbart, UW-Madison University Communications)
Data Sources: Internal
o AWS (Automatic Weather Station) datao Significant portion of internal data comes from UW’s AWS networko Real-time and archive quality data sets
o Ten minute intervalso Hourly intervalso Three-hourly intervals
o 60+ weather stations across the continent
Data Sources: Internalo Antarctic Composites
o Conceived by UW’s Professor Charles R. Stearns in October 1992.o Cover South Pole region to about 40° South.o Satellites used:GOES, GMS, MTSAT, METEOSAT, FY-2, KALPANA,
NOAA, DMSP, Aqua & Terra, and MetOp
o Created every 3 hours/1 hour every dayo Currently, 4 sizes:
o 512 x 512o 683 x 683o 1024 x 1024 (standard)o 2048 x 2048
o 6 types:o Infraredo Water Vaporo Experimental Visibleo Long-wave (Coming soon!)o Short-wave (Coming soon!)o Pseudo Color
Making Antarctic Composite Satellite
Imagery:
Data Sources: Internalo University of Wisconsin’s Space Science and Engineering Center
(SSEC) Data Centero Surface synoptic observationso METARs (Meteorological Aviation Reports)o RAOB (Radiosonde Observations)o Ship/buoy observations
o Numerical Models from GFS
(Global Forecast System) and ECMWF
(European Centre for Medium-Range
Weather Forecasts). o Polar-orbiting LAC (Local Area Coverage) observations from NOAA
satellites
Data Sources: Externalo Participation from various researchers, USAP (United States
Antarctic Program) participants, and international groups within the Antarctic meteorological community
o International AWS (real-time only)o USAP research vessel observationso Field experiment radiosonde observationso Surface observations at:
o Palmer Stationo McMurdo Stationo South Pole Stationo Various other field camps
Lawrence M. Gould. Photo by: Al Hickeyhttp://photolibrary.usap.gov/
Data Flow
International AWS
USAP Research Vessels
McMurdo, Palmer & South
Pole Stations
WisconsinAWS
AMRC Office
SSEC Data Center
NOAAHRPT
DMSPRTD
Aqua/Terra DB (McMurdo only)
NOAAGAC/LAC
MTSAT
Meteosat (2)
GOES (2)
FY-2C
Kalpana-1
Aqua/Terra NASA/NOAA
Products:
Data Formats
Archive:o McIDAS
o AREAo MDo GRID
o ASCII Texto GIF, JPEG
Distribution:o McIDAS
o AREAo MDo GRID
o ASCII Texto netCDF o GIF, JPEG
Data Archive: Historical
o Since 1992...o Media used to archive data:
1. Magneto-optic disks2. Compact disks (CD)3. Digital video disks (DVD)4. 8 millimeter Exabyte tapes5. DDS3 tapes (4 mm tapes)6. 3590 IBM tapes7. LTO tapes
1
2
3
4
5
6
7
Data Archive: Presento RAID/5 disk arrays as well as DVDs are most prominent todayo FTP site most widely used
o Updated monthly witho AWS – SPAWAR obso USAP research vessel observationso McMurdo Station obso Palmer Station obso South Pole Station obs
o Summary / notification email sent out
Distributiono Data Services to be discussed in Part II:
o FTP (File Transmission Protocol)o Rsync Serviceo Antarctic-IDD (Internet Data Distribution) LDM (Local Data Manager)o ADDE (Abstract Data Distribution Environment)o E-mailo Websiteo Future:
o RAMADDA (Repository for Archiving, Managing and Accessing Diverse Data)o GIS (Geographic Information Systems) Software
Challenges:1o USAP moving away from FTP site – advancing security ruleso Don’t fret! FTP will still be available here at the AMRC:
o Other options: Link with external systemo Limited resources to maintain linkso Manually providing data is expensive and not desired
o We aim to provide data in a more semi-automated, self-service fashion.
Challenges:2o Accessing data externally
o Poor qualityo Corrupt fileso Missing
o Gaps in the archiveo Which data can be used / distributed and which cannot?
o Third party redistribution limitations
o Evolving toward nearly fully automated data flow system (from source to archive)o Maintenance will be a constant issue – especially at the sourceo Documentation
Challenges:3 - Metadatao Ever increasing need for Metadatao Significant effort to research/document and provideo Incomplete historical records, leave some metadata information
missing…o E.g. South Pole...
We have found an issue with the South Pole data:
From 1996 and earlier, the South Pole LCDs (Local Climate Data) used data that switched between using local time and GMT time when recording observation times.
South Pole is working on re-running past LCDs and we do not know how long it will take / when they will be completed.
Acknowledgements
The authors appreciate the support of the Automatic Weather Station Program and/or Antarctic Meteorological Research Center for the data set, data display, information, etc. ([AMRC/AWS staff member, as appropriate], NSF grants number ANT-0636873, and/or ANT-0838834)
Sculptor Gabriel Warren used a transparency of the B-15A Iceberg in one of his pieces called “Interrogation: Dialogue” in his “Rhegmalogia” collection.
http://www.art-farm.net/homeGabe.htm
AMRC/AWS Statistics
o Show Gigabytes transferred / month on Website.o Average of 70-80 Gb year-round.
o Gigabytes transferred / month on FTP site.o Generally very variable – maybe due to field season.
• ANTIDD – Antarctic-IDD mailing list• ANTDISCUSSION – Antarctic meteorology
discussion group• ANTAWS - Information and updates on
AWS data• ANTOBS - Information and updates on all
other observational datasets• ANTOPS - Notices on AMRC’s real-time
data availability, distribution and status• ANTICEBERG - Notices on latest Iceberg
monitoring images posted to the web site• ANTCOMMUNITY - Infrequent notices to
the Antarctic community on events and significant developments
Please join our mailing lists!:
How to subscribe:Visit our new website and click on the “Subscribe” button
next to the mailing list you wish to join:
THANK YOU!
Acknowledgements:
National Science Foundation Office of Polar Programs Grant #0838834South Pole Meteorology Office/Raytheon Polar ServicesSPAWAR Office of Polar ProgramsMcMurdo Station Research AssociatePalmer Station Research AssociateUSAP Research Vessels – LM Gould & NB PalmerMcMurdo Ground StationInternational partners& More
Questions?