go-essp workshop, llnl, livermore, ca, jun 19-21, 2006, center for atmosphere sciences and...
TRANSCRIPT
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Construction of e-science Environment
for Meteorological Data in Korea
Jai-Ho Oh Dept. Environmental Atmospheric Science
Pukyong National University
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Contents
• Introduction of the meteorological data sharing system.– Background– Concept– Architecture of the meteorological data sharing system
• Meteorological Information Search/Display system.
• Conclusion and Future Works
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Introduction of the meteorological data sharing system
• Background– Too many types of meteorology data is produced everyday with
big size.– Difficulties to find meteorology data that scientists required to
model.
• Concepts– If we have a metadata of meteorology data in indexed DB from
the SAM file stored in SAN, we can easily find meteorology data.– If we control sharing permission of each meteorology data, we
can safely handle and share the meteorology data.– If we are able to display the meteorology data before download ,
we can decide if the searched data is really one we need.
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Development of Meteorological Data Sharing System
• Construction of Internal Data Grid Basis using Super Computer– Construction of Data Grid middleware
Resource Broker Data management : Replica Service Information and Monitoring Service Large amount storage service Security
– Preparation of User Interface for Conjugation of Data Grid
• Design and Preparation of Meta catalog for Numerical Simulations– Design Metadata of KMA NWP data following WMO metadata Standards– Construction of NWP data sharing prototype using Metadata catalog– Meta Catalog
Metadata Presentation Metadata Update Data Formats Support of data transport protocols Support of data access protocols Authentication and Authorization Advanced World Meteorology Information Network V-GISC and Logical Harmony of Metadata
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Detail objectives of Meteorological Data Sharing System
• Data Management Service: Simple way to search and acquire Selected Meteorological data (sorted by Area and Time) - Development of platform for WMO data sharing - Compatible with Data Grid
• Construction of Data Pool - Shared Data server using Data broker - Development of Common Interface for Atmospheric data server as like LAS (Live Access Server) & GDS (GrADS Data Server)
• Metadata cataloging Service - Prototype design for Metadata of Meteorological data - DataBase of Metadata
• Output Display Service - Various Visualization techniques to interpret Meteorological model results - GridSphere based on GrADS Plug-in
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Structure of meteorological Data sharing system
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Architecture of the meteorological data sharing system
• Architecture of the meteorology data sharing system– Consist of Two systems.
Meteorological Administration CenterMeteorology information search/display system
Meteorological
Administration center
Meteorology information
display system
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Access control of meteorological data(1)
Select meteorology data we want to change Permission and Information.
Show all meteorology data
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Access control of meteorological data(2)
Click Enable to approve
sharing and add additional
Info
Click Disable to prevent
sharing
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Manage the information of meteorological data
Click Meta-Info to see description
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Search Agent configuration
Search Configuration
DB
Setting the search condition
keep the search condition in DB
Search AgentSearch meteorological data using search condition in DB
Search Condition1. File Type2. Production time3. Execution time
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Work flow of the search agent
WGRIB
grib2ctl
Control FileSearch Agent Register Agent DBMS
1. Search meteorological data to register2. Make a Control File for Register Agent
3. Register meta data of meteorological data to DBMS
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
History view of the search request
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Register data grid server
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Meteorological information display system
• Software used by the meteorology information display system– GridSphere– Globus Toolkit– GrADS
• Functions of the meteorology information display system– Seek meteorology data using search condition.– Show the list of results.– Setting the GrADS display options.– Show meteorology data by graphic.– Users can have their own DataBox so that they can keep the se
arched results for later use.
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Structure of the meteorological information display system
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Legacy System Integration Efforts
GDS&LASData Broker
• Implemented Data Broker for GDS&LAS• Search Agent invokes Data Broker to add new datasets into the configuration file• Run ‘Restart’ script to affect recently added datasets
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Work Flow between GridSphere and USER
Authentication using GT
Connect to the web server of Gridsphere Job Creation
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Search Option Page
Input the Search Condition and Push the Search button
Show the search result
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Setting the GrADS option
Setting the GrADS optionAnd push Show Graph button
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Image Output(1)
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Image Output(2)
GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006,
Center for ATmosphere sciences and Earthquake Researches
Conclusion & Future Work
• Efficient Way for Data Sharing for Administrator
• Agent Based Data Searching/Cataloging
• Integration with Grid Sphere (portlet)
• On-Demand Service Architecture
• Dynamic Meta Data Cataloging
• WSRF based Data Sharing
• Extension to various types of data
• Need more collaboration with other nations’ weather centers
• Any contributions to the E-Community