high performance cluster computing architectures and systems
DESCRIPTION
High Performance Cluster Computing Architectures and Systems. Hai Jin. Internet and Cluster Computing Center. Multiple Path Communication. Introduction Heterogeneity in Networks and Applications Multiple Path Communication Case Study Summary and Conclusion. Introduction (1). - PowerPoint PPT PresentationTRANSCRIPT
High Performance Cluster Computing
Architectures and Systems
Hai JinHai Jin
Internet and Cluster Computing CenterInternet and Cluster Computing Center
2
Multiple Path Communication
Introduction Heterogeneity in
Networks and Applications
Multiple Path Communication
Case Study Summary and
Conclusion
3
Introduction (1)
Communication is major factor of performance in network-based computing
Utilize all available network resources within a system Substantial benefits in simultaneously
exploiting multiple independent communication paths between processors
Heterogeneous network environment Single application execution Different network - different performance
characteristic
4
Introduction (2)
Performance-based path determination (PBPD) Exploiting multiple physical communication paths &
multiple communication protocols within a single parallel application program to reduce communication overhead
Different physical networks & protocols have different performance characteristics
Different types of communication within an application might be best suited to one of several different communication paths
bulk data transfer (image data): – high-bandwidth communication
short control messages (synch, acknowledgement): - low-latency communication
5
Introduction (3)
Performance-based path determination (PBPD) Performance-based path selection (PBPS)
Dynamically select the best communication path among several in a given system
Performance-based path aggregation (PBPA) Aggregate multiple communication paths into a single
virtual path for sending an individual message in an application
Each independent path simultaneously carries a fraction of a single message
Useful in bandwidth-limited situations Control is based on message size, message type, or
network traffic load
6
Heterogeneity in Network and Applications
Heterogeneity inherent interprocessor communication
Different types of messages latency-limited bandwidth-limited
7
Varieties of Communication Networks
WAN (1 Mbps) MAN (100 Mbps) SAN (more than 1 Gbps) Ethernet (10 Mbps – 1000 Mbps) ATM (155 Mbps) FDDI (100 Mbps) HiPPI (800 Mbps – 1.6 Gbps) Infinitband (10Gbps)
8
Exploiting Multiple Communication Paths
Messages size communication groups priority level
Different type of network services to support different types of message communication
Network services connection oriented connectionless
Both are logically sufficient for the implementation of any communication pattern
Each offers performance & programming benefits for some classes of applications
9
Benefits to Support Multiple Communication Paths
Efficient network utilization Most appropriate communication paths to be
used Alternative network protocols
Connection oriented / connectionless Robust communication
Multiple networks provides extra reliability Network load balancing Quality of service
Multiple paths can support diverse QoS requirements
10
Multiple Path Communication
A single application is likely to require several different types of messages for communication Different types of messages may be better
suited to a different type of communication mechanism
11
Performance-Based Path Selection (PBPS)
Useful when one provides better performance in one situation while the other is better in another situation
Possible to dynamically select the appropriate communication path for a given communication event
f1(m1) = t1 f2(m2) = t2 fPBPS(m) = Best[fi(m)], where (i= 1.. N)
12
When sending a message of size mi, performance-based path selection (PBPS) uses the lower latency curve among f1(mi) and f2(mi)
13
Performance-Based Path Aggregation (PBPA)
Can be applied when different paths show similar characteristics 2 nearly identical networks
aggregate 2 networks into a single virtual network bandwidth will be nearly twice
Divide – transmit - aggregate Important consideration
determine the size of the submessages
fPBPA(m) = fi(mi), where (i= 1.. N), (m = mi)
14
When using performance-based path aggregation (PBPA) with two networks, a message of size m1+ m2 is split into two submessages
such that messages of size m1 and m2 are sent over networks f1(mi) and f2(mi) simultaneously
15
PBPD Library
Custom library whose main feature is the support of multiple communication paths in a single application program
Based on common TCP layer Add integer field ‘length’ tells message size
Used by PBPA if a message is too small to segment Handle the multiplexing of different TCP
connections Use UNIX select system call with appropriate table
lookups
16
Implementation and Protocol Hierarchy of the PBPD
Communication Routines
17
Case Study: Multiple Path Characteristic
Communication type in parallel application program point-to-point, collective
4 Silicon Graphics Challenge L shared-memory multiprocessors. 4 or 8 R10000 processors at 196MHz per node 10Mbps Ethernet and 266Mbps Fiber Channel
18
Multiple Heterogeneous Network Configuration used in the
Experiments
19
Point-to-point and Broadcast Characteristics of Ethernet using the TCP and UDP
Communication Protocols
20
Example of a Broadcast Operation of a Separate Addressing Method using the PBPA
Technique
21
Point-to-point Characteristics of Ethernet and Fibre Channel using the TCP Protocol
22
Broadcast Characteristics of Ethernet and Fibre Channel using the TCP Protocol
23
Case Study: Communication Patterns of Parallel Applications
Performance of PBPD at application-level depends on the communication patterns of the specific application being executed
24
Parallel Benchmark Programs Tested
Programs Description
CG Conjugate gradient
MG Multigrid
IS Integer sort
Filter Smoothing (averaging) filter
Gauss Gaussian elimination
Hough Line recognition algorithm
Kirsch Image processing
TRFD Two-electron integral transformation
Warp Spatial domain image restoration
BT Simulated CFD application using Block tridiagonal solver
LU Simulated CFD application using LU solver
SP Simulated CFD application using Pentadiagonal solver
MICOM Miami isopycnic coordinate ocean model
25
Case Study: Computation Model
Data parallelization Medium to coarse-grained parallelism Similar communication pattern for every
node except when starting application
Each processor tends to alternate computation and communication at the same time communication congestion is inevitable
26
Relative Times of Communication Events for the IS Benchmark
27
Relative Times of Communication Events for the MG Benchmark
28
Case Study: Message Size and Destination Distributions
Distribution of message destinations Uniformly distributed among all nodes Biased to some destinations for each node
Distribution of message sizes
29
Overall Message Destination Distribution for All of the Test Programs
30
The Message Destination Distribution for Each Processor in the CG Benchmark
31
The Cumulative Distribution of Message Sizes in the Test Programs
32
Experiments Results
Parameterize communication pattern Point-to-point communication
Small message mean size l1, probability to appear is b
Large message mean size l2, probability to appear is 1 - (b+c)
Broadcast communication Message mean size l3, probability to appear is c
Assumption 1 master, p-1 slave processors Message size for each communication follow
Poison distribution with 3 different mean value l1, l2, l3
33
Parameter Values Used in the Synthetic Benchmark
Applicationtype
A
B
C
pt2pt (small)mean l1
pt2pt (large)mean l2
broadcastmean l3
b 1-(b+c) c
45% 45% 10%
25% 25% 50%
5% 5% 90%
34
Speedup using PBPS with the TCP and UDP Protocols over Ethernet. The speedups are normalized to the
case when using the TCP protocol alone
35
Speedup using PBPA Technique with Ethernet and Fiber Channel using the TCP Protocols. The speedups are
normalized to the case when using the Ethernet alone
36
Summary and Conclusion
Communication overhead can be reduced by exploiting heterogeneity in both communication path and application Reduce communication overhead
PBPD technique can achieve performance improvement based on message type