scalable group communications and systematic group modeling
DESCRIPTION
Scalable Group Communications and Systematic Group Modeling. Jun-Hong Cui University of Connecticut [email protected] http://www.cse.uconn.edu/~jcui. Cool Application 1 : Teleconferencing. Cool Application 2 : Telemedicine. Cool Application 3 : Net Gaming. Multicast: What and How?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/1.jpg)
Scalable Group Communications and Systematic Group Modeling
Jun-Hong Cui University of Connecticut
[email protected] http://www.cse.uconn.edu/~jcui
![Page 2: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/2.jpg)
Cool Application 1 : Teleconferencing
![Page 3: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/3.jpg)
Cool Application 2 : Telemedicine
![Page 4: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/4.jpg)
Cool Application 3 : Net Gaming
![Page 5: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/5.jpg)
Jun-Hong Cui (c) UCONN 2004
Multicast: What and How?
Multicast: One to many or many to many communications (group communications)
To achieve multicast: Multiple unicast (one to one) Network multicast---IP multicast Overlay multicast (using proxies) Application layer multicast (end host)
![Page 6: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/6.jpg)
Jun-Hong Cui (c) UCONN 2004
Outline of this talk
Scalable Group Communications--- Aggregated Multicast
Systematic Group Modeling--- GEM Model
Research Directions
![Page 7: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/7.jpg)
Jun-Hong Cui (c) UCONN 2004
IP Multicast Group: IP D class address
Use Tree delivery structure
Routers: keep forwarding entries per-group/source (multicast state)
IP multicast Resource efficient Scalable to group size
Customer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
group NHop
g1 Ab, A3
![Page 8: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/8.jpg)
Jun-Hong Cui (c) UCONN 2004
The Problem: Not Scalable to the Number of Groups
More groups more trees
More forwarding entries More tree maintenance
overhead IP multicast NOT scalable
to the number of groups State Scalability problem Serious in transit domains
Our solution Aggregated multicast to
improve state scalabilityCustomer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
group NHop
g1 Ab, A3
g2 Ab, A3
![Page 9: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/9.jpg)
Jun-Hong Cui (c) UCONN 2004
Key Insight
There are many overlaps among multicast trees in transit domains
Customer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
group NHop
g1 Ab, A3
g2 Ab, A3
![Page 10: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/10.jpg)
Jun-Hong Cui (c) UCONN 2004
Aggregated Multicast Key idea:
Force multiple groups share a single delivery tree (aggregated tree)
Benefits: Reduce state at core
routers Reduce tree
maintenance overhead
Push complexity to edge
Target: Multicast provisioning
in transit domainsCustomer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
Tree NHop
T1 Ab, A3
![Page 11: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/11.jpg)
Jun-Hong Cui (c) UCONN 2004
Aggregated Multicast (cont.)
Core routers: Keep state per-tree
Edge routers: Keep group state
Groups: Aggregate at
incoming edge router De-aggregate at
outgoing edge routers
Customer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
Tree NHop
T1 Ab, A3
Aggregation
De-aggregation
De-aggregation
![Page 12: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/12.jpg)
Jun-Hong Cui (c) UCONN 2004
Perfect Match vs. Leaky Match
Group-Tree match Perfect match Leaky match
Bandwidth waste in leaky match Data delivery to non-member nodes
Customer Networks, Domain D
D1
Domain X
Domain A
Domain C
Domain Y
X1
Ab
AaA1
A2Domain B
B1
A3
C1Y1
Tree NHop
T1 Ab, A3
Discard Packets
![Page 13: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/13.jpg)
Jun-Hong Cui (c) UCONN 2004
Aggregation Control
Leaky match Good for tree aggregation But waste bandwidth
There is a trade-off Static group-tree matching: NP hard A dynamic group-tree matching algorithm to control the trade-off Under a given bandwidth waste threshold
![Page 14: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/14.jpg)
Jun-Hong Cui (c) UCONN 2004
Group-Tree Matching
Domain X
Domain A
Customer Networks, Domain D
Domain C
Domain Y
X1
AbAa
D1
A1
A4
Domain B
B1
A3
C1Y1
Domain E E1 A2
![Page 15: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/15.jpg)
Jun-Hong Cui (c) UCONN 2004
Group-Tree Matching
Domain X
Domain A
Customer Networks, Domain D
Domain C
Domain Y
X1
AbAa
D1
A1
A4
Domain B
B1
A3
C1Y1
Domain E E1 A2
![Page 16: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/16.jpg)
Jun-Hong Cui (c) UCONN 2004
Implementation Issues Multiplex multiple groups over a shared tree
IP encapsulation MPLS (Multi-Protocol Label Switching)
Tree management and group-tree matching Tree Manager (need to know group membership) Distributed or centralized solutions
Have designed and implemented protocols: ASSM for source specific multicast (SSM) BEAM for shared tree multicast (ASM) AQoSM for QoS multicast provisioning
![Page 17: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/17.jpg)
Jun-Hong Cui (c) UCONN 2004
Extend to Overlay and Adhoc Net
Overlay multicast Implement multicast in overlay net
A collection of proxies (or gateways) Processing power, memory & bandwidth more critical
Aggregated multicast reduces management overhead
Wireless multicast Implement multicast in wireless adhoc net
No infrastructure, self-organized Energy, memory, bandwidth, resilience very critical Aggregated trees help to improve performance
![Page 18: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/18.jpg)
Jun-Hong Cui (c) UCONN 2004
Overlay Network
![Page 19: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/19.jpg)
Jun-Hong Cui (c) UCONN 2004
Adhoc Network
![Page 20: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/20.jpg)
Jun-Hong Cui (c) UCONN 2004
Outline of this talk
Scalable Group Communications--- Aggregated Multicast
Systematic Group Modeling--- GEM Model
Research Directions
![Page 21: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/21.jpg)
Jun-Hong Cui (c) UCONN 2004
The Problem: Group Modeling
The locations of the group membersGiven a graph, where should we place them?
Current assumptions: uniform random model (unproven)All members uniformly distributed Not realistic for many applications
![Page 22: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/22.jpg)
Jun-Hong Cui (c) UCONN 2004
Group Modeling is Critical
Some studies have shown the locations of members have significant effects on Scaling properties of multicast trees Aggregatability of multicast state Performance of state reduction schemes
Realistic group models Improve effectiveness of simulation Guide the design of protocols
![Page 23: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/23.jpg)
Jun-Hong Cui (c) UCONN 2004
Our Contributions
Measure real group membership properties MBONE (IETF/NASA) and Netgames (Quake)
Design a model to generate realistic membership GEneralized Membership Model (GEM) Use Maximum Enthropy: a statistical method
![Page 24: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/24.jpg)
Jun-Hong Cui (c) UCONN 2004
Roadmap
Membership Characteristics Measurement and Analysis Results
Model Design and Validation
![Page 25: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/25.jpg)
Jun-Hong Cui (c) UCONN 2004
Beyond Uniform Random Model
How close are the members in a group?
Are all the members same in group participation?
What are the correlations between members in group participation?
![Page 26: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/26.jpg)
Jun-Hong Cui (c) UCONN 2004
An Illustration (Teleconference)
Internet
Edge Router Member Router
Seattle
Boston
AtlantaLos Angeles
0.5
0.5
0.7
0.4
1.0
![Page 27: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/27.jpg)
Jun-Hong Cui (c) UCONN 2004
Membership Characteristics
Member clustering Capture proximity of group members Use network-aware clustering method
Group participation probability Show difference among members/clusters
Pairwise correlation in group participation Capture joint probability of two members/clusters Use correlation coefficient (normalized covariance)
![Page 28: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/28.jpg)
Jun-Hong Cui (c) UCONN 2004
Measure Membership Properties
MBONE applications (from UCSB) IETF-43 (Audio and Video, Dec. 1998) NASA Shuttle Launch (Feb. 1999) Cumulative data sets on MBONE (1997-1999)
Net Games (using QStat) Quake I (query master server) Choose 10 most popular servers (May. 2002)
Examine three membership properties
![Page 29: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/29.jpg)
Member Clustering
CDF of cluster size for MBONE and net games
MBONE cumulative data sets
MBONE real data sets
Net game data sets
(3, 0.64)
![Page 30: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/30.jpg)
Group Participation Probability
CDF of participation probability for Net Game data sets
![Page 31: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/31.jpg)
Group Participation Probability
CDF of participation probability for MBONE applications
![Page 32: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/32.jpg)
Pairwise Correlation in Group Participation
CDF of correlation coefficient for Net Game data sets
![Page 33: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/33.jpg)
Pairwise Correlation in Group Participation
CDF of correlation coefficient for MBONE applications
![Page 34: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/34.jpg)
Jun-Hong Cui (c) UCONN 2004
Generalized Membership Model--- GEM (An Overview)
Network topologyCluster methodGroup behavior
Distr. of participation prob. Distr. of pairwise correlation Distr. of member cluster size
1. Create clusters in given topology2. Select clusters as member clusters According to input distributions
3. Choose nodes for each member clusters
Desired number of multicast groupsthat follow the given distributions
Inputs
GEM
Outputs
![Page 35: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/35.jpg)
Jun-Hong Cui (c) UCONN 2004
Member Distribution Generation
Definition: K clusters: C1 , C2 , … , Ci , … , CK
Prob. pi for any i in [1, K]
Joint prob. pi,j for any i, j in [1, K]
X=(X1 ,X2 , … , Xi , … , Xk): Xi is a binary indicator
Xi = 1 if Ci is in the group
Xi = 0 if Ci is not in the group Objective:
Generate vectors x=(x1 , x2 , … , xk) satisfying
P(Xi = 1) = pi and P(Xi = 1 , Xj = 1) = pi,j
![Page 36: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/36.jpg)
Jun-Hong Cui (c) UCONN 2004
Maximum Entropy Method
To calculate the distribution of (X1,X2, …, Xk) requires O(2K) constraints
But we only know O(K+K2) constraints We use Maximum Entropy Method
Entropy is a measure of randomness We construct a maximum entropy distr. p*(x)
Satisfy constraints in specified dimensions Keep as random as possible in unconstrained
dimensions i.e. maximize entropy while match given constraints
![Page 37: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/37.jpg)
Jun-Hong Cui (c) UCONN 2004
Three CasesConsidering P(Xi=1)= pi and P(Xi=1, Xj=1)=pi,j
1. Uniform distr. without correlation (easy) pi,j = pi * pj , and pi = pj
2. Non-uniform distr. without correlation (easy) pi,j = pi * pj , but pi = pj not necessary
3. Non-uniform distr. with pairwise correlation Neither pi,j = pi * pj nor pi = pj necessary
Need to calculate the maximum entropy distr. p*(x)
Entropy decreases from case 1 to case 3
![Page 38: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/38.jpg)
Jun-Hong Cui (c) UCONN 2004
Experimental Validation
Consider all membership properties
Consider three cases Figures omitted …Our experiments show
GEM can regenerate groups satisfying given distributions (from real measurement)
![Page 39: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/39.jpg)
Jun-Hong Cui (c) UCONN 2004
Summary Uniform random model
Can capture net games approximately But not realistic for MBONE applications
GEM: a generalized membership model Three cases (case 1: uniform random model) Realistic membership can be regenerated
Beyond multicast Peer-to-peer network modeling
Beyond wired networkWireless adhoc networks, sensor networks …
![Page 40: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/40.jpg)
Jun-Hong Cui (c) UCONN 2004
Outline of this talk
Scalable Group Communications--- Aggregated Multicast
Systematic Group Modeling--- GEM Model
Research Directions
![Page 41: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/41.jpg)
Jun-Hong Cui (c) UCONN 2004
Networking: Expanding Visions
(from Jim Kurose)
![Page 42: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/42.jpg)
Peer-to-Peer Networking
![Page 43: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/43.jpg)
Peer-to-Peer NetworkingFocus at the application level
![Page 44: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/44.jpg)
Jun-Hong Cui (c) UCONN 2004
Applications & Challenges Applications
P2P file sharing (Napster, Gnutella, Freenet, etc.) Application-layer multicast
Characteristics each node potentially same responsibility, functionality
logical connectivity rather than physical connectivity Why P2P?
High resource utilization (bandwidth, memory, CPU) Challenges
Self-organized and large scale (routing) Reliability and security
![Page 45: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/45.jpg)
Jun-Hong Cui (c) UCONN 2004
Research Directions Overlay multicast
Scalability, QoS, security, pricing, … Multicast modeling
Systematic multicast evaluation Peer-to-peer networks
measurement & modeling, complex queries Wireless adhoc networks
Mobility modeling, scalable multicast Sensor networks
Sensor deployment and security Very large scale sensor network design
![Page 46: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/46.jpg)
Jun-Hong Cui (c) UCONN 2004
http://www.cse.uconn.edu/~jcui
![Page 47: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/47.jpg)
Jun-Hong Cui (c) UCONN 2004
THANKS!!!
![Page 48: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/48.jpg)
Jun-Hong Cui (c) UCONN 2004
Network Characteristics No fixed infrastructure, instantly deployable Node portability, mobility Error-prone channel Limited resources
bandwidth, energy supply, memory and CPU. Heterogeneous nodes
big/small; fast/slow etc Heterogeneous traffic
voice, image, video, data Wireless multihop connection
to save power, overcome obstacles, enhance spatial spectrum reuse, etc
![Page 49: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/49.jpg)
Jun-Hong Cui (c) UCONN 2004
Calculate the Maximum Entropy Distribution
dxxpxpxp logmax arg*
jiwhenpdxxpxx jiji ,,
ii pdxxpx
The maximum entropy distr. p*(x) is the solution for:
1 dxxp
Subject to
and and
Use lagrange multipliers and numerical method to construct p*(x), Then Gibbs Sampler to sample it
![Page 50: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/50.jpg)
Group Participation Probability
Participation probability distribution for IETF43-Video
![Page 51: Scalable Group Communications and Systematic Group Modeling](https://reader033.vdocuments.net/reader033/viewer/2022051215/56814902550346895db6341c/html5/thumbnails/51.jpg)
Pairwise Correlation in Group Participation
Joint probability distribution for IETF43-Video