high data volume transfer issues at noaa
DESCRIPTION
High Data Volume Transfer Issues at NOAA. Christopher D. Elvidge Earth Observation Group National Oceanic and Atmospheric Administration National Geophysical Data Center Boulder, Colorado August 26, 2005 [email protected]. NOAA’s Vision and Mission. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/1.jpg)
High Data Volume Transfer Issues at
NOAA
Christopher D. ElvidgeEarth Observation Group
National Oceanic and Atmospheric Administration
National Geophysical Data CenterBoulder, ColoradoAugust 26, 2005
![Page 2: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/2.jpg)
2
NOAA’s Vision and Mission
• VISION – An informed society that uses a comprehensive understanding of the role of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions.
• MISSION – To understand and predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet our Nation’s economic, social, and environmental needs.
![Page 3: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/3.jpg)
3
Benefits of Satellite Earth Observations
Energy Resources
Natural & Human Induced Disasters
Human Health & Well-Being
Climate Variability & Change
Water Resources
Weather Information, Forecasting &
Warning
Terrestrial, Coastal & Marine
Ecosystems
Sustainable Agriculture &
Desertification
Biodiversity
![Page 4: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/4.jpg)
4
NOAA’s Current Satellite Observing Systems
• Polar-orbiting Operational Environmental Satellites (POES) ~10 GB per day.
• Geostationary Operational Environmental Satellites (GOES) ~ 10 GB per day.
• Defense Meteorological Satellite Program (DMSP) ~ 8 GB per day.
![Page 5: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/5.jpg)
5
NOAA’s Future Satellite Observing Systems
• NPOESS Preparatory Project (NPP) ~4 TB per day 2008- 2012.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C1 ~4 TB per day 2010-2015.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C2 ~4 TB per day 2012-2017.
• National Polar-orbiting Operational Environmental Satellite System (NPOESS) C3 ~4 TB per day 2014-2019.
• Geostationary Operational Environmental Satellites-R (GOES-R) ~ 2 TB per day 2012-2017.
![Page 6: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/6.jpg)
6
Projected NOAA Data Holdings By Year
![Page 7: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/7.jpg)
The NOAA Comprehensive Large Array-data Stewardship System(CLASS) will be the long term
archive for the data from these systems, providing data access for
the scientific community.
![Page 8: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/8.jpg)
8
Hardware and Network Design
• CLASS will have three operational sites (NGDC, NCDC, and NASA IV&V)
• CLASS will have a centralized development environment and a centralized integration and test environment at NSOF
• CLASS will use a Multi-protocol Label Switching (MPLS)-based peerless IP network used for site to site communication
• New hardware and network architecture– Implemented by January 2006– Modular, scalable, and redundant– Addresses security requirements
![Page 9: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/9.jpg)
9
CLASS Multi-Site ConfigurationCLASS Multi-Site Configuration
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
CISCOSYSTEMS
Fairmont
DMZ
AIX Nodes
Web ServersBoulder
AshevilleData Provider(e.g., IDPS)
Suitland
DMZWeb Servers
AIX Nodes
AIX Nodes
DMZWeb Servers
DMZ
Web Servers
AIX Nodes
CISCOSYSTEMS
Cisco 7606 Router
Cisco 6509 w/ FWSM
OC-12
SX-MM Fiber
Copper GE
MPLS
![Page 10: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/10.jpg)
10
CLASS Distribution Process
![Page 11: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/11.jpg)
11
NGDC will have direct access to the NLR via the Front Range GigaPop
for use in delivering data from CLASS
![Page 12: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/12.jpg)
12
The Data Hog?
The Visible-Infrared Imaging Radiometer Suite (VIIRS) will fly on NPP and each of the NPOESS satellites.
Raw data (RDRs) from a VIIRS will run 116 GB per day.
Radiance calibrated geolocated data (SDRs) will tilt to 815 GB per day.
Environmental data records (EDRs) will bulge towards 2 TB per day.
![Page 13: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/13.jpg)
13
The Challenges
Moving the data within NOAA - from the Suitland to the CLASS nodes.
Providing simple, useable tools to establish and manage large volume data subscriptions.
Transferring large volumes of data to the users, including those in the APAN region.
MAFFIN, Chiba U., U. Tokyo, Tokoku U. in Japan have expressed interest in receiving global VIIRS data!
![Page 14: High Data Volume Transfer Issues at NOAA](https://reader036.vdocuments.net/reader036/viewer/2022062802/5681449a550346895db1458b/html5/thumbnails/14.jpg)
14
Thanks for Listening
QUESTIONS?