4ª Reunión Red Ibérica MM5

The Internet Data Distribution SystemThe Internet Data Distribution System

Group 17presenter: Guilherme O. Chagas (goc@ufrj.br)

IDD Network• IDD – Unidata Internet Data Distribution System– Disseminating near real-time earth observations via the

Internet– Open to participation by universities and degree-granting

institutions for purpouses of education and research– Real-time data delivery system: data sent as soon as

available from the observing system– “Subscription” based data delivery: users specify in advance

which data should be delivered to their system– Community based architecture: hierarchical data distribution

scheme

IDD Network• Organized and guided by the Unidata Program Center

(UPC). – “Community of over 160 institutions vested in the common

goal of sharing data, and tools to access and visualize that data”

– A program of UCAR/UOP, sponsored by NSF– Acts as the leader of a community with over 160 institutions– Vested in the commom goal of sharing data and tools to

access and visualize earth-related data

www.unidata.ucar.edu

IDD Network• Community driven– UPC:

• develops, maintains and supports many tools at no charge• coordinates the usage of high-interest data• acts as a data facilitator

– Universities:• maintain computers and infrastructure to serve the community as

defined by UPC• Inject data of less interest in IDD without UPC involvement

IDD Network• Members in the US include:– National Center for Environmental Prediction (NCEP)– National Weather Service (NWS)– Storm Prediction Center (SPC)– Tropical Prediction Center (TPC)– Aviation Weather Center (AWC)– Hydrologic Prediction Center (HPC)– Marine Prediction Center (MPC), etc...

IDD-Brazil effort• IDD-Brazil: extension of IDD network through South

America– Started at LPM/UFRJ in 2001– Officially created in 2003 – Meteoforum– Joint effort involving:

• UFRJ• CPTEC• USP• UFPA• Unidata

– Constant interaction with Universities in Portugal

How the network operates• LDM – Local Data Manager: the IDD building block– Collection of programs that select, capture, manage and

distribute arbitrary data– Network client and server abilities, specific protocol– Event-Driven data distribution: allowing real-time distribution– Extremely configurable and flexible

How the network operates

LDM communications scheme

How the network operates• A worldwide community– Multiple sources: each LDM server can receive data from

many sites and distribute it to many others– Redundancy and reliability: the IDD system is decentralized,

has no datacenter– Load distribution: designed to avoid excessive traffic on any

node or link– Requires only an Internet connection: worlwide reach

How the network operates

IDD topology

Available Data• Types of data provided in Real-time (high-interest)

– Domestic Data Service (e.g., surface, upper air, buoy) (US only) – Public Products Service (textual information; e.g., watches, warnings,

forecast discussions, etc.) (US only) – International Data Service (worldwide observations) – Operational forecast model output (e.g., GFS, NAM, UKMET, etc.)– GOES Satellite imagery and derived products – Aircraft Data (ACARS/AMDAR) Information – NOAA Profiler Network data – High-resolution model data from NCEP (e.g., GFS 0.5 degree)– WSR 88-D NIDS and Level II radar products (US only)– Canadian Meteorological Center GEM model output – NOGAPS and COAMP model data from FNMOC – National Lightning network data – GPS Precipitable water vapor observations

Available Data

CPTEC/ETA sample plots

Available Data• Product visualization and generation using GEMPAK– Collection of programs capable of decoding received data– Generates graphics and analysis– NMAP2: GUI that aggregates several programs in a unified

interface– Flexible: based on tables (indexes) to configure its

processes

Available Data

Satellite derived winds in NMAP2

IDD in Portugal• Universidade de Aveiro: using IDD since 2003– Operational usage of NCEP’s GFS (including 0.5 degree) to

run WRF and MM5.– Project submitted to FCT, planned to provide training

workshops to scientific community.

• Universidade de Évora: distributing data to Africa– SICLIMAD: providing training to Portuguese speaking

countries.– Cooperation with CPTEC/INPE.

Summary• IDD: a community-driven network• Freely available for Universities and Degree-granting

institutions• Delivers reliable, real-time data• Expansible: users can add data with virtually no costs• Provides excellent tools to visualize and generate

products• Enhancing cooperation between different countries