slide 1 et-cts - gisc network bandwidth requirementsslide 1 gisc network bandwidth requirements a...
TRANSCRIPT
ET-CTS - GISC Network Bandwidth Requirements
Slide 1
Slide 1
GISC network bandwidth requirements
A proposed method for an educated guess…
Hiroyuki Ichijo – JMA
Rémy Giraud - ECMWF
ET-CTS Co-Chairs
ET-CTS - GISC Network Bandwidth Requirements
Slide 2
Slide 2
Introduction As part of the requirements of the WIS, each GISC must:
host a Discovery, Access and Retrieval (DAR) catalogue hold and distribute at least 24 hours of current WMO data and
products intended for global distribution.
Each GISC is responsible for coordinating with the Centres within its area of responsibility a WIS telecommunications infrastructure that can meet the WIS requirements for information exchange within the area and that can exchange agreed WIS time critical and operational critical information.
The WIS real-time network would be composed of a small number of Area Meteorological Data Communication Networks (AMDCNs) and a WIS core network interconnecting the GISCs and AMDCNs together. GISCs would be responsible for ensuring that their AMDCNs are developed and maintained and, where the appropriate agreements are in place, the traffic between WIS centres in different AMDCNs is managed. [extract from CBS-XIV Abridged final report ]
ET-CTS - GISC Network Bandwidth Requirements
Slide 3
Slide 3
Main taskRequired GISC bandwidth on the core network should be considered carefully considering not only bulk but also peak traffic. In addition, it is important to study smooth evolution process in gradual participation of operational GISCs.
Background for consideration (1) The Improved MTN (IMTN) project started at ET-IMTN meeting in
Sep/Oct 1999. Its implementation has progressed in technical evolution process as the configuration of two clouds (i.e. Network I & Network II) until Sep 2009. By the end of 2009, all links via Network I have migrated to via Network II. Currently the IMTN is operating on a single coordinated MPLS cloud.
Since a WIS core network will be established on the IMTN, it is easy to realize the full-mesh GISC topology. However it is expected that traffic on the WIS core network handled by GISCs will considerably increase. To support future-oriented WIS operation, network capacity of each GISC will have to be expanded in collaboration with others.
ET-CTS - GISC Network Bandwidth Requirements
Slide 4
Slide 4
Background for consideration (2) Appropriate maximum number of GISCs has come up for discussion
repeatedly since the initial stage called as FWIS:[Extraction from the final report of CBS-ext.02, Annex IV]Several (perhaps four to 10) centres would serve as GISCs. Each GISC would have a defined area of responsibility. GISCs would usually be located within or closely associated with a centre running a global data assimilation system or having some other global commitment, such as a WMC.
[Extraction from ET-CTS outcome reported to ICG-WIS-3 in 2006]Correlation between the number of GISCs and reasonableness of full-mesh topology of a WIS core network: From the practical and relative evaluation, the full-mesh can be appropriate on the assumption that the number of GISCs would be less than 7 inclusive. In case of more GISCs, the full-mesh should be avoided.
However 13 GISCs candidates have been identified as of the end of November 2009.
Sense of a possible basisFurther study is desired to clarify the appropriate maximum number of GISCs from the practical view of network bandwidth requirements.
ET-CTS - GISC Network Bandwidth Requirements
Slide 5
Slide 5
Hypothesis (1) Flows
Each GISC will receive data from its area of responsibility on an AMDCN
A backup GISC should be available to receive data from Centres from another AMDCN
All GISCs will be linked directly each other (i.e. mesh-topology)
Each GISC will push all data received to all other GISCs
Not all AMDCN will provide the same amount of data. (i.e. unbalanced model)
ET-CTS - GISC Network Bandwidth Requirements
Slide 6
Slide 6
Hypothesis (2) Technical
A WIS core network to link all GISCs will not be based on multi-cast but uni-cast mechanism for, at least, initial implementation
All network protocol have an overhead. A 20% overhead is a typical value for FTP
Despite control, there will be duplicated data over the network.
In order to define the bandwidth requirements, it will be necessary to estimate:
Daily data volume to be sent and received
Estimate of peak rates if the data transfers are not evenly
spread over the 24 hours.
ET-CTS - GISC Network Bandwidth Requirements
Slide 7
Slide 7
Points to study (1) The required bandwidth for each GISC will depend on:
The total number of GISCs
The estimated amount of data to be transferred daily by each GISC
The potential peaks of traffic within the 24 hours period
The traffic repartition between AMDCN:
An unbalanced traffic pattern between the GISCs will have a lot of impact
The protocols used on the network and their overhead
Thus we should: i) estimate data volume for global exchange more practically; ii) develop a matrix sheet to indicate bandwidth requirements in each case; iii) recommend an appropriate maximum number of GISCs if possible.
ET-CTS - GISC Network Bandwidth Requirements
Slide 8
Slide 8
Points to study (2) An online spreadsheet available at
http://spreadsheets.google.com/ccc?key=0AsDEZG6XKnasdG9MNWlRYXUzeVJJRkJsWFppWlI0UEE&hl=fr
Is a tool to be used to automate some calculation for the bandwidth. Based
on values to be defined online by the user, it is then possible to estimate
the required bandwidth.
To access this document a freely available Google account is required
Please use Sheet “Comments and History” to add comments and
suggestions to improve the spreadsheet.
Slide 9
References and hints
ET-CTS - GISC Network Bandwidth Requirements
Slide 10
Slide 10
IMTN evolution
Exeter
SofiaMelbourne
Buenos Aires
TokyoBeijin
g
Nairobi
Washington
Network INetwork I
Jeddah
Prague
Toulouse
Dakar Algiers
Offenbach
Brasilia
Cairo
New Delhi
Moscow
Network IINetwork II
Exeter
Sofia
Melbourne
Buenos Aires
TokyoBeijing
Nairobi
Washington
Jeddah
Prague
Toulouse
Dakar Algiers
Offenbach
Brasilia
Cairo
New Delhi
IMTN cloud
Moscow
RA I
RA II
RA III
RA IV
RA V
RA VI
Exeter
Sofia
Melbourne
Buenos Aires
TokyoBeijing
Nairobi
Washington
Jeddah
Prague
Toulouse
Dakar Algiers
Offenbach
Brasilia
Cairo
New Delhi
IMTN cloud
Moscow
RA I
RA II
RA III
RA IV
RA V
RA VI
Exeter
Sofia
Melbourne
Buenos Aires
TokyoBeijing
Nairobi
Washington
Jeddah
Prague
Toulouse
Dakar Algiers
Offenbach
Brasilia
Cairo
New Delhi
IMTN cloud
Moscow
RA I
RA II
RA III
RA IV
RA V
RA VI
ET-CTS - GISC Network Bandwidth Requirements
Slide 11
Slide 11
WIS multicast network InternetInternet
IGDDS,etIGDDS,etcc
Full-mesh topology of WIS core network Full-mesh topology will bring two benefits: (1) maximum redundancy in the configuration of GISC backup channels;(2) operational simplicity in data synchronization
GTSGTSIMTNIMTN
WIS core network
More than 180 centers
Max 18 centers
Less than 13 centers
AMDCN
AMDCN
AMDCN
Conceptual interrelation between GTS and WIS networks
DARDAR
AMDCN
ET-CTS - GISC Network Bandwidth Requirements
Slide 12
Slide 12
AMDCN
Priorities of study for bandwidth requirements
WIS core
networkAMDC
NGISC
InternetInternetIGDDS,etIGDDS,etcc DARDAR
First prioritySecond priority
Third priority
Target in this period
ET-CTS - GISC Network Bandwidth Requirements
Slide 13
Slide 13
Architecture and flows
ET-CTS - GISC Network Bandwidth Requirements
Slide 14
Slide 14
Balanced model (4 GISCs case)
WIS core network
(Full-meshed topology)
Responsible area #1
AMDCN #1
GISC #1
2GB Responsible area #2
AMDCN #2
GISC #2
2GB
Responsible area #3
AMDCN #3
GISC #3
2GB Responsible area #4
AMDCN #4
GISC #4
2GB
2GB2GB
2GB2GB
2GB2GB
Incoming
daily volume
Outgoing
daily volume
Necessary
port speed
GISC #1
6 GB 6 GBBandwidth
appropriate
for 6GB/day
GISC #2
ditto ditto ditto
GISC #3
ditto ditto ditto
GISC #4
ditto ditto ditto
Example case of total volume of 8GB
ET-CTS - GISC Network Bandwidth Requirements
Slide 15
Slide 15
WIS core network
(Full-meshed topology)
Responsible area #1
AMDCN #1
GISC #1
5GB Responsible area #2
AMDCN #2
GISC #2
1GB
Responsible area #3
AMDCN #3
GISC #3
Responsible area #4
AMDCN #4
GISC #4
1GB1GB
1GB1GB
1GB5GB5GB
5GB
1GB1GB1GB5GB
1GB
1GB
Unbalanced model (4 GISCs case)
Incoming
daily volume
Outgoing
daily volume
Necessary
port speed
GISC #1
3 GB 15 GB
Bandwidth
appropriate
for 15GB/day
GISC #2
7 GB 3 GBBandwidth
appropriate
for 7GB/day
GISC #3
ditto ditto ditto
GISC #4
ditto ditto ditto
Example case of total volume of 8GB
ET-CTS - GISC Network Bandwidth Requirements
Slide 16
Slide 16
Internal 77.81%
FTP 18.01% Sockets 3.81%X.25 0.35%
Async 0.02%
HTTP 0.00%
Email 0.00%
Reference to volume required (1)
Average daily volume received at RTH Washington: 26.66 GB (as of May 2008)
Volume received by Protocol
ET-CTS - GISC Network Bandwidth Requirements
Slide 17
Slide 17
Internal 8.86%FTP 8.50%
Sockets 81.97%
X.25 0.36%
Async 0.31%
Reference to volume required (2)
Volume transmitted by Protocol
Average daily volume transmitted from RTH Washington: 30.81 GB (as of May 2008)
ET-CTS - GISC Network Bandwidth Requirements
Slide 18
Slide 18
Reference to volume required (3)
Results of RTH data survey 1 August 2007
RTH-A RTH-B RTH-C RTH-D Average Min Max Daily volume (messages) 855 MB 2 GB Daily volume (files) 986 MB 0.55 GB Total daily volume (GB) 1.8 2.6 4.52 2.92 2.96 1.8 4.52
Daily number of messages 202,200 289,784 369,065 397,021 314517.5 202200 397021
Incoming (including duplicated ones)
Daily number of files 425 567 1,540 30,727 8314.75 425 30727
Daily volume (messages) 545 MB 2.4 GB
Daily volume (files) 571 MB 1.6 GB
Total daily volume (GB) 1 4 0.753 6.89 3.18575 0.753 6.89
Daily number of messages 292500 855,318 238,747 1,135,909 630618.5 238747 1135909
GTS
Outgoing (after duplicated check and
elimination)
Daily number of files 526 951 539 47,098 12278.5 526 47098
Total daily incoming traffic (GB) 146 2.6 41.57 8.47 49.66 2.6 146 Download volume
Daily obtaining volume on operational basis 1.8 GB 13 MB
Total daily outgoing traffic (GB) 137 13.9 148.43 19.18 79.6275 13.9 148.43 Internet
Upload volume Daily providing volume on operational basis 2.3 GB 1.1 GB
Incoming Total daily volume (GB) 0.2 24.8 8.1 0.01 8.3 0.01 24.8 Other connections Outgoing Total daily volume (GB) 0.1 62.5 8.3 0.3 17.8 0.1 62.5
Growth Rate Per Annum in volumes 12% 32-50% 35-50%
Extract from ICG-WIS IV (31.VIII.2007) /Doc. 3.1(1)http://www.wmo.int/pages/prog/www/ISS/Meetings/ICG-WIS_ECMWF2007/documents/RRR-Doc3-1-1.doc
ET-CTS - GISC Network Bandwidth Requirements
Slide 19
Slide 19
Reference to volume required (4)
Past and projected data volumes (RTH-C)
Extract from ICG-WIS IV (31.VIII.2007) /Doc. 3.1(1)http://www.wmo.int/pages/prog/www/ISS/Meetings/ICG-WIS_ECMWF2007/documents/RRR-Doc3-1-1.doc