noaa’s nws telecommunication gateway rth washington icm mtn et-oi, sep 2008 office of the chief...
Post on 02-Jan-2016
215 Views
Preview:
TRANSCRIPT
NOAA’sNWS Telecommunication Gateway
RTH WashingtonICM MTN ET-OI , Sep 2008
NOAA’sNWS Telecommunication Gateway
RTH WashingtonICM MTN ET-OI , Sep 2008
Office of the Chief Information Officer
NOAA’s National Weather Service
Office of the Chief Information Officer
NOAA’s National Weather Service
“Taking the pulse of the planet”“Taking the pulse of the planet”
Data Input to the NWSTGData Input to the NWSTG
MPLS
IP / Sockets
X.25
Asynchronous
FTP – http://weather.gov/tg/ftpingest.html
Email – http://weather.gov/tg/emailingest.html
Web - http://weather.gov/tg/bullguid.html
MPLS
IP / Sockets
X.25
Asynchronous
FTP – http://weather.gov/tg/ftpingest.html
Email – http://weather.gov/tg/emailingest.html
Web - http://weather.gov/tg/bullguid.html
“Taking the pulse of the planet”“Taking the pulse of the planet”
Replacement & Backup RTHsReplacement & Backup RTHs
Full functional replacement of existing capabilities
Expanded capacity and capability
– Input data growth from 200 to 800 GB/day
– Output data growth from 800 to 2400 GB/day
Transition to new technology
– Middleware for internal transport
– Network-centric systems interconnectivity
– Central switching engine with relational database
– SAN and NAS storage solutions
Highly scaleable architecture
Hardware refresh
Full functional replacement of existing capabilities
Expanded capacity and capability
– Input data growth from 200 to 800 GB/day
– Output data growth from 800 to 2400 GB/day
Transition to new technology
– Middleware for internal transport
– Network-centric systems interconnectivity
– Central switching engine with relational database
– SAN and NAS storage solutions
Highly scaleable architecture
Hardware refresh
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTG/BTG PerformanceRTG/BTG Performance
Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds
Routine Message Latency (averaged monthly) 60 secondsDaily Traffic Volume (averaged monthly) 1.2TB
Typical Performance Results (averaged monthly)
Performance Category Threshold ActualSystem Availability 99.90% 100%Warning Message Latency 10 seconds < 1 secondRoutine Message Latency 60 seconds < 1 secondDaily Traffic Volume 1.2TB 1.2 - 1.5TB
Performance Metric Threshold System Availability (averaged monthly) 99.90% Warning Message Latency (averaged monthly) 10 seconds
Routine Message Latency (averaged monthly) 60 secondsDaily Traffic Volume (averaged monthly) 1.2TB
Typical Performance Results (averaged monthly)
Performance Category Threshold ActualSystem Availability 99.90% 100%Warning Message Latency 10 seconds < 1 secondRoutine Message Latency 60 seconds < 1 secondDaily Traffic Volume 1.2TB 1.2 - 1.5TB
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTH WashingtonRTH Washington
Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent
Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day
On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS)
– GTS delivers tsunami data and warnings to connected MTN centers within two minutes
On AFTN network (KWBCYMYX) & ICAO OPMET Databank
Can route more than 50 routine messages per second with reliability for all dissemination to all of its users of 99.9 percent
Latency for high priority traffic of 10 seconds or less and routinely disseminates 1.2 terabytes of information per day
On Main Trunk Network (MTN) of the WMO Global Telecommunication System (GTS)
– GTS delivers tsunami data and warnings to connected MTN centers within two minutes
On AFTN network (KWBCYMYX) & ICAO OPMET Databank
“Taking the pulse of the planet”“Taking the pulse of the planet”
NOAANet Logical DesignNOAANet Logical Design
“Taking the pulse of the planet”“Taking the pulse of the planet”
Primary / BackupNWSTG Access
Primary / BackupNWSTG Access
“Taking the pulse of the planet”“Taking the pulse of the planet”
MS3
FEP #1Sockets
FEP #2X.25
FEP #8 MSD
FEP #3FTP
FEP #4Async
FEP #7Marta
FEP #6eMail
FEP #5Web
MS3 #1
#1
MS3
#2
#3
#4#8
#7
#6
#5
MS3 #2
MS3 #3
MS3 #4
MS3 #5
MS3 #6
MS3 #7
MS3 #8
Legend
Front EndProcessor (FEP)
MS3 Hub
Process FlowsWebSphere MQ Integrator
MS3
#1
MS3 inbound queue
MS3 customer queue
WebSphere MQ Queues
WeatherData
Source/Customers
Weather ProductPrefix
MS3 Message
Logical cluster queue
Weather Product
Weather ProductPrefix
“Taking the pulse of the planet”“Taking the pulse of the planet”
Major Software ComponentsMajor Software Components
Core software components of the NWSTG replaced in 2006
Data switching software
– Core software rewritten and implemented in IBM WebSphere Message Broker
– Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services
Data transport infrastructure (middleware)
– Shared data stores replaced with IBM MQSeries Middleware
– Applications send data to each other without concern for target location
– Data delivery guaranteed even during failures
Data storage infrastructure (relational database)
– All data stores moved into Sybase relational database
Core software components of the NWSTG replaced in 2006
Data switching software
– Core software rewritten and implemented in IBM WebSphere Message Broker
– Message Broker provides an application execution environment, threading, and fourth generation language support (ESQL), integrated with middleware services
Data transport infrastructure (middleware)
– Shared data stores replaced with IBM MQSeries Middleware
– Applications send data to each other without concern for target location
– Data delivery guaranteed even during failures
Data storage infrastructure (relational database)
– All data stores moved into Sybase relational database
“Taking the pulse of the planet”“Taking the pulse of the planet”
Internal RedundancyInternal Redundancy
Internal redundancy implemented for critical processes
Active – Warm Standby
– Standby system running at all times, system disks moved to standby system during failover
– EXAMPLES: Individual front-end processors, application servers
Active – Hot Standby
– Data replicated to Hot Standby system in realtime, automatic failover
– EXAMPLE: Sybase relational database servers
Active - Active
– Multiple systems active in parallel, data shared/routed between systems groups
– EXAMPLES – Switching system, HTTP/FTP server groups
Internal redundancy implemented for critical processes
Active – Warm Standby
– Standby system running at all times, system disks moved to standby system during failover
– EXAMPLES: Individual front-end processors, application servers
Active – Hot Standby
– Data replicated to Hot Standby system in realtime, automatic failover
– EXAMPLE: Sybase relational database servers
Active - Active
– Multiple systems active in parallel, data shared/routed between systems groups
– EXAMPLES – Switching system, HTTP/FTP server groups
“Taking the pulse of the planet”“Taking the pulse of the planet”
Architecture OverviewArchitecture Overview
“Taking the pulse of the planet”“Taking the pulse of the planet”
Architecture OverviewArchitecture Overview
“Taking the pulse of the planet”“Taking the pulse of the planet”
System ExtensibilitySystem Extensibility
Virtualized server hardware platform selected for extensibility
Virtualization
– System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame
– Individual resources selected to create virtual servers
– Major storage implemented in Storage Area Network
– Virtual disks allocated to virtual servers
Extensibility
– Unused resources can be allocated to virtual servers, as needed
– Underused resources can be removed from a virtual server
– Virtual disks can be extended or reduced as storage requirements change
Virtualized server hardware platform selected for extensibility
Virtualization
– System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.) available within hardware frame
– Individual resources selected to create virtual servers
– Major storage implemented in Storage Area Network
– Virtual disks allocated to virtual servers
Extensibility
– Unused resources can be allocated to virtual servers, as needed
– Underused resources can be removed from a virtual server
– Virtual disks can be extended or reduced as storage requirements change
“Taking the pulse of the planet”“Taking the pulse of the planet”
System ExtensibilitySystem Extensibility
Virtualized server hardware platform selected for extensibility
Advanced Virtualization
– Network and SAN I/O interfaces can be shared through virtual I/O
– CPUs can be carved into fractional components down to 1/10th of a CPU
Advanced Extensibility
– Resources (CPU, Memory) included within system that were not purchased
– If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available
Virtualized server hardware platform selected for extensibility
Advanced Virtualization
– Network and SAN I/O interfaces can be shared through virtual I/O
– CPUs can be carved into fractional components down to 1/10th of a CPU
Advanced Extensibility
– Resources (CPU, Memory) included within system that were not purchased
– If additional resources are needed, resources are “turned on” via software key after purchase and are immediately available
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTH Washington WIS approach
RTH Washington WIS approach
RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements
– Production and Replicated core systems
• All functions not essential to mission critical operations (i.e. switching) execute against replicant
– SOA Approach to design
– Integral file, message and application switching
– RDBS core design
• ~ 2000 destinations defined
• ~ 400K defined products
• ~ 10M entries (rows) in switching table
– Many tables used for managing bulletin, file and report storage including dissemination and services metadata.
RTG/BTG upgrades made with knowledge of coming WIS/GISC requirements
– Production and Replicated core systems
• All functions not essential to mission critical operations (i.e. switching) execute against replicant
– SOA Approach to design
– Integral file, message and application switching
– RDBS core design
• ~ 2000 destinations defined
• ~ 400K defined products
• ~ 10M entries (rows) in switching table
– Many tables used for managing bulletin, file and report storage including dissemination and services metadata.
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTH Washington WIS approach
RTH Washington WIS approach
We view Data Access & Retrieval (DAR) service as the only new service provision
– Plan to lay this service onto existing architecture
– Will break up DAR functions
• DAR catalog creation & update
• discovery & access servicing
– DAR will not execute on primary production system (at least in early stages of implementation
We view Data Access & Retrieval (DAR) service as the only new service provision
– Plan to lay this service onto existing architecture
– Will break up DAR functions
• DAR catalog creation & update
• discovery & access servicing
– DAR will not execute on primary production system (at least in early stages of implementation
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTH Washington WIS approach
RTH Washington WIS approach
Develop metadata catalog first in conjunction with national & regional partners
– Will leverage knowledge expertise of partners
– Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners.
– Work will be made available to all Members
– Will collaborate with WIS Project Team, WMO ETs and developers in other Regions
– Regional WIGOS Development Project will meet WIS requirements
• RA-IV Integrated Atmosphere Observing System
• With initial emphasis on upper-air observations
Develop metadata catalog first in conjunction with national & regional partners
– Will leverage knowledge expertise of partners
– Will expand to all RTH (GISC) data holdings and holdings of appropriate Centers (RSMC/DCPC/NC) and willing partners.
– Work will be made available to all Members
– Will collaborate with WIS Project Team, WMO ETs and developers in other Regions
– Regional WIGOS Development Project will meet WIS requirements
• RA-IV Integrated Atmosphere Observing System
• With initial emphasis on upper-air observations
“Taking the pulse of the planet”“Taking the pulse of the planet”
Meteorological Assimilation Data Ingest System (MADIS) Transition to Operations
SFC-LANDSFC-MARINE
U/A-IN SITU
U/A-REMOTESENSING
SATELLITE
GRIDSMETADATA
MADISCollection,
QC, andDistribution
InformationBases
(QCed Datasets)
Quality ControlInformation
(Data QC Flags)And Metadata
PM 12/12/05
“Taking the pulse of the planet”“Taking the pulse of the planet”
MADIS Ingests and Performs QC on Meteorological Observational DataMADIS Ingests and Performs QC on Meteorological Observational Data
Meteorological Surface
– METAR
– Airways
– Maritime
– Modernized NWS Cooperative Observer
– UrbaNet
– Integrated Mesonet
• State DOT
• Mesonets
• AWS
RSAS Surface Grids
Meteorological Surface
– METAR
– Airways
– Maritime
– Modernized NWS Cooperative Observer
– UrbaNet
– Integrated Mesonet
• State DOT
• Mesonets
• AWS
RSAS Surface Grids
NOAA Profiler Network
Hydrological Surface
Automated Aircraft
Multi-Agency Profiler
Cooperative Agency Profiler
Radiosonde
Radiometer
Satellite Wind
– NOAA GOES Products
Satellite Radiances and Soundings
– NOAA POES
Snow
NOAA Profiler Network
Hydrological Surface
Automated Aircraft
Multi-Agency Profiler
Cooperative Agency Profiler
Radiosonde
Radiometer
Satellite Wind
– NOAA GOES Products
Satellite Radiances and Soundings
– NOAA POES
Snow
“Taking the pulse of the planet”“Taking the pulse of the planet”TOC Environment
Gateway
MADIS-T
FTP
Ex
tern
al I
SP
LDM
FTP
Storage(SAN)
Automated QC processing
Gateway Customers (NCDC, NCF, GTS etc)
Data Processing
LDM
NOAA Customers
Non-NOAAObservations
LDM
Integrated Datasets
Web Server
Internet
Web
Ser
ver
?
?
?Individual Data
Products
Tunnel
“Taking the pulse of the planet”“Taking the pulse of the planet”
RTH Washington Other Issues
RTH Washington Other Issues
– Foundation of Programs is Observing Systems & Data Exchange
– Transition of GTS-MTN
• Two existing clouds to ???
– Cloud 1 - frame relay
– RMDCN – satellite MPLS
– Addition of Guam/Hono to AP-RARS
– Future operations locations
• Backup first then Primary
– Foundation of Programs is Observing Systems & Data Exchange
– Transition of GTS-MTN
• Two existing clouds to ???
– Cloud 1 - frame relay
– RMDCN – satellite MPLS
– Addition of Guam/Hono to AP-RARS
– Future operations locations
• Backup first then Primary
© Crown copyright Met Office
GTS Link with NOAA-TOC Washington
Inbound flow may be inhibited due to the saturated Outbound flow being unable to return IP sockets ‘ack’ packets quickly enough.
“Taking the pulse of the planet”“Taking the pulse of the planet”
NWSTG Functional OverviewNWSTG Functional Overview
The NWSTG is:
the central communication facility of the NWS;
the primary acquisition and distribution center for NWS data and products;
the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;
A major data exchange hub for NOAA and other agency data and products.
The NWSTG is:
the central communication facility of the NWS;
the primary acquisition and distribution center for NWS data and products;
the primary acquisition and distribution center for international data and products to meet WMO, ICAO and bi-laterally agreed US requirements;
A major data exchange hub for NOAA and other agency data and products.
“Taking the pulse of the planet”“Taking the pulse of the planet”
NWSTG Functional OverviewNWSTG Functional Overview
The NWSTG includes
WMO Regional Telecommunication Hub (RTH) Washington
ICAO OPMET Databank
the ASOS Operations and Monitoring Center
• Operational oversight of U.S. federal automated surface observing systems
the AWIPS Network Control Facility
• AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement
• The NCF is the central communications hub and technical support center for AWIPS
The NWSTG includes
WMO Regional Telecommunication Hub (RTH) Washington
ICAO OPMET Databank
the ASOS Operations and Monitoring Center
• Operational oversight of U.S. federal automated surface observing systems
the AWIPS Network Control Facility
• AWIPS is the main NWS’ system which supports the NWS forecast and warning mission requirement
• The NCF is the central communications hub and technical support center for AWIPS
“Taking the pulse of the planet”“Taking the pulse of the planet”
Dissemination Systems Dissemination Systems
GTS
ISCS
EMWIN
Internet
GMDSS
NWS Telecom. Gateway
GTS
ISCS
EMWIN
Internet
GMDSS
NWS Telecom. Gateway
NWWS
NWR
LDAD
Family Of Services (FOS)
Interagency Connections
AWIPS SBN / NOAAPORT
NWWS
NWR
LDAD
Family Of Services (FOS)
Interagency Connections
AWIPS SBN / NOAAPORT
International National
“Taking the pulse of the planet”“Taking the pulse of the planet”
Dissemination and Distribution
NWSTG
FOS
SBN/NOAAPORT• GOES •NCEP Product Suite
NWWS(10 sec)
WAN
NESDIS
Internet-BasedDissemination
EMWIN(> 1 min)
Telecommunications Operations Center
Satellite Legend:EMWIN
NOAAPORT
NWWS
Commercial Satellite
GOES
NWRWSR-88D
Field Offices
NCEP
DataServers
NCF
Media &Other
Customers
Public
SpecializedCustomers
Other
Agencies
LDAD
Designated Local
Customers
•Commercial Weather Services•Research Institutions•FAA, etc.
ISCS
GTS EMWIN
Private line Dissemination
• Observations
Imagery
WMO
ICAO
“Taking the pulse of the planet”“Taking the pulse of the planet”
Geographically separatedbackup system
Replacement / BackupTG Description
Replacement / BackupTG Description
Worldwide
users• ICAO / WMO• Govt. agencies• Family of
services• Internet users• Foreign
countries• Emergency
mgrs
NWS users• Nat’l Centers for Environ. Prediction• NWS Regional Offices• Domestic/int’l observation & forecast offices• AWIPS
NWS Telecom Gateway• Located in Silver Spring• New message switchingsystem allows future upgrades• 2x upgradeability• Much improved response time • Redundancy ensuresuninterrupted service• Full configuration management
120GB/46600 GB
80GB/38200 GB
90GB/46375 GB
700GB/382000 GB
Daily Throughput/# of circuitsLegacy over Replacement
“Taking the pulse of the planet”“Taking the pulse of the planet”
System ExtensibilitySystem Extensibility
“Taking the pulse of the planet”“Taking the pulse of the planet”
Government Networking Requirements - NOAANetGovernment Networking Requirements - NOAANet
• IP-based networking solution
− Any-to-any connectivity
− High degree of bandwidth scalability
− Optimum redundancy and survivability
− IP convergence (i.e., voice, video and data over IP)
− High-end performance
• Network security remains paramount, particularly in light of today’s socio-political threats
• Segmentation from the public Internet
• Minimizes risk of security or privacy breaches
“Taking the pulse of the planet”“Taking the pulse of the planet”
Sprint Peerless IP Network MapSprint Peerless IP Network Map
Internet Transport Node
OC48 (significant portions upgrading to OC192)
ChicagoChicago
AnaheimAnaheim
San JoseSan Jose Pennsauken, NJPennsauken, NJ
Kansas CityKansas City
StocktonStockton CheyenneCheyenne
FortWorthFort
Worth
New YorkNew York
TacomaTacoma
AtlantaAtlanta
Relay, MDRelay, MD
DCDC
top related