enhancing neighborship consistency for peer-to-peer distributed virtual environments

1

Enhancing Neighborship Consistency for Peer-to-Peer

Distributed Virtual Environments

Jehn-Ruey Jiang, Jiun-Shiang Chiou and Shun-Yun HuDepartment of Computer Science and Information Engineering

National Central University

2

Outline

• Introduction

– Background

– P2P DVEs

• Factors affecting Neighborship Consistency

• Proposed Solutions

• Simulation Results

• Conclusion

3

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

4

DVE (1)

• Distributed Virtual Environments (DVEs) are computer-generated virtual world where multiple geographically distributed users can assume virtual representatives (or avatars) to concurrently interact with each other

• A.K.A. Networked Virtual Environments (NVEs)

5

DVE (2)

• Examples of DVEs include early DARPA SIMNET and DIS systems, as well as currently booming Massively Multiplayer Online Games (MMOGs).

6

Adaptive Computing and Networking Lab, CSIE, NCU

Massively Multiplayer Online Games

• MMOGs are growing quickly – 8 million registered users for World of Warcraft– Over 100,000 concurrent players– Billion-dollar business

7


8


11


12

DVE (3)

• 3D virtual world with – People (avatar)– Objects– Terrain– Agents– …

• Each avatar can do a lot of operations – Move– Chat– Using items– …

13

Issues for DVE

• Scalability– To accommodate as many as participants

• Consistency– All participants have the same view of object states

• Persistency– All contents (object states) in DVE need to exist persistently

• Reliability– Need to tolerate H.W and S.W. failures

• Security– To prevent cheating and to keep user information and game

state confidentially.

14


The Scalability Problem (1)

• Client-server: has inherent resource limit

Resource limit

[Funkhouser95]

15


The Scalability Problem (2)

• Peer-to-Peer: Use the clients’ resources

Resource limit

[Keller & Simon 2003]

16


You only need to know some participants

★: self

▲: neighbors

Area of Interest

(AOI)

17

Neighborship Consistency (1)

• Definition # current AOI neighbors observed

# current AOI neighbors

18

:is observed neighbor

Neighborship Consistency (2)

• An example

Neighborship Consistency

= 4 / 5

= 80%

:is actual neighbor

19

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

20

Related Work (1):DHT-based: SimMUD

• B. Knutsson, H. Lu, W. Xu and B. Hopkins, “Peer-to-peer Support for Massively Multiplayer Games,” in Proceedings of INFOCOM 2004.

• Authors are fromDepartment of Computer and Information Science, University of Pennsylvania

21

Related Work (1):DHT-based: SimMUD

[Knutsson et al. 2004] (UPenn)

• Pastry (DHT mapping) + Scribe (Multicast)

• Fixed-Sized Regions

• Coordinators

22

SimMUD -- Introduction

• Proposes use of P2P overlays to support Massively multiplayer games (MMG)

• Primary contribution of paper:– Architectural (P2P for MMG)– Evaluative

23


PASTRY(P2P overlay)

SCRIBE(Multicast support)

MMG GAME

24


• Players contribute memory, CPU cycles and bandwidth for shared game state

• Persistent user state is centralized– Example: payment information, character

• Allows central server to delegate to peers the dissemination and the process of intensive game states

25

Distributed Game Design

• GAME STATES

– Game world divided into connected regions

– Regions are controlled by different coordinates

26


27


• Game design based on fact that:– Players have limited movement speed– Limited sensing capability– Hence data shows temporal and spatial

localities– Use Interest Management

• Limit amount of state player has access to

28


• Players in same region form interest group

• State updates relevant to group disseminated only within group

• Player changes group when going from region to region

29


• GAME STATE CONSISTENCY– Must be consistent among players in a region– Basic approach: employ coordinators to resolve

update conflicts– Split game state management into three classes to

handle update conflicts:• Player state• Object state• The Map

30


• Player state– Single writer multiple reader– Position change is most common event

• Use best effort multicast to players in same region• Use dead reckoning to handle loss or delay

31


• Object state– Use coordinator-based mechanism for shared

objects– Each object assigned a coordinator– Coordinator resolves conflicting updates and

keeps current value

32


• Map– Maps are considered read-only because they

remain unchanged during the game play.– They can be created offline and inserted into

the system dynamically.– Dynamic map elements are handled as

objects.

33

Game on P2P overlay

• Map game states to players– Group players & objects by region– Map regions to peers using pastry Key– Each region is assigned ID– Live Node with closest ID becomes coordinator– Random Mapping reduces chance of coordinator

becoming member of region (reduces cheating)– Currently all objects in region coordinated by one

Node– Could assign coordinator for each object

34

Game on P2P overlay

• Shared state replication– Lightweight primary- backup to handle failures– Failure detected using regular game events– Dynamically replicate coordinator when failure

detected– Keep at least one replica at all times– Uses property of P2P (route message with

key K to node ID, say N , closest to K)

35

Game on P2P overlay

• Shared state replication (contd..)– The replica kept at M which is the next closest

to key K– If new node T added which is closer to K than

coordinator N• Forwards messages to coordinator N until all

states of K are transferred from N to T• Takes over as coordinator and N becomes a

replica

36

Game on P2P overlay

• Catastrophic failure– Both coordinator and replica dead

– Problem solved by cached information from nodes interested in the area

37

Experimental Results

• Prototype Implementation of “SimMud”• Used FreePastry (open source)• Maximum simulation size constrained by

memory to 4000 virtual nodes• Players eat and fight every 20 seconds• Remain in a region for 40 seconds• Position updates every 150 millisec by mul

ticast

38


• Base Results– No players join or leave– 300 seconds of game play– Average 10 players per region– Link between nodes have random delay of 3-

100 ms to simulate network delay

39

Experimental Results(Base results)

40


41


42


• 1000 to 4000 players with 100 to 400 regions

• Each node receives 50 –120 messages

• 70 update messages per second– 10 players * 7 position updates

• Unicast and multicast message take around 6 hops (but 50 hops in the worst case)

43


• Breakdown of type of messages– 99% messages are position updates– Region changes take most bandwidth– Message rate of object updates higher than

player-player updates• Object updates multicast to region• Object update sent to replica• Player player interaction effects only players

44


• Effect of Population Growth– As long as average density remains same,

population growth does not make difference

• Effect of Population Density– Ran with 1000 players , 25 regions– Position updates increases linearly per node– Non – uniform player distribution hurts

performance

45


• Three ways to deal with population density problem– Allow max number of players in region– Different regions have different size– System dynamically repartitions regions with

increasing players

46


• Effect of message aggregation– Since updates are multicast, aggregate them

at root– Position update aggregated from all players

before transmit– Cuts bandwidth requirement by half– Nodes receive less messages

47


48


49


• Effect of network dynamics– Nodes join and depart at regular intervals– Simulate one random node join and depart

per second– Per-node failure rate of 0.06 per minute– Average session length of 16.7 minutes (close

to 18 minutes for a FPS game -- Half Life)– Average message rate increased from 24.12

to 24.52

50

Related Work (2):Neighbor-list Exchange

[Kawahara et al. 2004] (Univ. of Tokyo)

• Fully-distributed• Nearest-neighbors• List exchange

• High transmission• Overlay partition

51

Related Work (3): Mutual Notification: Solipsis

[Keller & Simon 2003] (France Telecomm R&D)

• Links with AOI neighbor• Mutual cooperation• Inside convex hull

• Potentially slow discovery• Inconsistent topology

52

Related Work (4): P2P Hybrid (DHT, super-nodes,neighbor list exchange: MOPAR

• Yu, A. and Vuong, S. T., “MOPAR: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games,” In Proceedings of the international Workshop on Network and Operating Systems Support For Digital Audio and Video (Stevenson, Washington, USA, June 13 - 14, 2005). NOSSDAV '05.

53

Related Work (5): Voronoi-based Overlay Network

: VON

• SY Hu, JF Chen, TH Chen, “VON: a scalable peer-to-peer network for virtual environments,” IEEE Network, 2006

54

Design Goals

• Observation: – for virtual environment applications, the contents we want

are messages from AOI neighbors– Content discovery is a neighbor discovery problem

• Solve the Neighbor Discovery Problem in a fully-distributed, message-efficient manner.

• Specific goals:– Scalable

Limit & minimize message traffics– Responsive

Direct connection with AOI neighbors

55

Voronoi Diagram

• 2D Plane partitioned into regions by sites, each region contains all the points closest to its site

Neighbors

Site

Region

56

Design Concepts– Each node constructs a Voronoi of its neighbors– Identify enclosing and boundary neighbors– Mutual collaboration in neighbor discovery

Use Voronoi to solve the neighbor discovery problem

i

● node i and the big circle is its AOI■ enclosing neighbors▲ boundary neighbors

★ both enclosing and boundary neighbors▼ normal AOI neighbors

◆ irrelevant nodes

57

Procedure (JOIN)

1)Joining node sends coordinates to any existing node

Join request is forwarded to acceptor

2)Acceptor sends back its own neighbor list

joining node connects with other nodes on the list

Acceptor’s region Joining node

58

Procedure (MOVE)

1)Positions sent to all neighbors, mark messages to B.N.

B.N. checks for overlaps between mover’s AOI and its E.N.

2)Connect to new nodes upon notification by B.N.

Boundary neighbors

New neighbors

59

Procedure (LEAVE)

1)Simply disconnect

2)Others then update their Voronoi

new B.N. is discovered via existing B.N.

Leaving node (also a B.N.) New boundary neighbor

60

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

61

Factors affecting Neighborship Consistency

• Mobility– Speed

AOI-radius

• Node failure– Possibly causes overlay partition

62

Mobility

63

Overlay Partition (1/2)

• Nodes divide into two or more groups

• Each group can not be aware each other

• May destroy neighborship consistency

• In pure P2P DVE– May be impossible to detect and recover– Have to avoid it

64

Overlay Partition (2/2)

×

×

×

×

×

×

65

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

66

Proposed Solutions

• Adaptive AOI– AOI radius increases with speed increasing

• Detecting critical nodes– Critical nodes are the nodes in some regions

where the density is low– It is used to avoid overlay partition

67

Adaptive AOI

: AOI buffer : Central node : AOI

Central node is of speed d

t Adaptive AOI buffer is of width d

’

68

Critical Nodes (1/4)

69


• Periodically detect by node itself # AOI neighbors Neighbor_

level = Average # AOI neighbors

• If a node detects that its neighbor level is lower than a pre-specified threshold, it regards itself as a critical node.

• A critical node will perform the backup operation to send to all connected neighbors the backup message.

70


• A Backup message– Sent by critical nodes– Contains a neighbor list of the critical node

• Stock neighbor list– Stores nodes from backup message– Uses to periodically check

• Check operation– Check all nodes in its stock neighbor list periodically– To discovery missed neighbors

71


A

BC

…

…

…

…

…

…

…

…

…

Neighbor list of node A

Stock neighbor list of node A

…

…

…

…

Neighbor list of node B

B

C

C

72

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

73

Simulation Parameters

• 1000 nodes• in a 1200-unit by 1200-unit plane• constant velocity of 5 units per time-step• AOI radius is 100 units (the number of directly

connected neighbors is limited to be 20)• Clustered distribution: nodes cluster into three

groups in the first 300 steps and can move randomly after 300 steps.

• Random distribution: nodes can move randomly in all simulation steps.

74

Mobility (1/2)

75

Mobility (2/2)

76

Node Failure (1/8)

77

Node Failure (2/8)

78

Node Failure (3/8)

79

Node Failure (4/8)

80

Node Failure (5/8)

81

Node Failure (6/8)

82

Node Failure (7/8)

83

Node Failure (8/8)

84

Outline

• Introduction

– Background

– P2P DVEs




• Conclusion

85

Conclusion

• Address two factors– Node mobility– Node failure

• Improve neighborship consistency by– Adaptive AOI buffer– Critical node detection

86

Q&A

enhancing neighborship consistency for peer-to-peer distributed virtual environments

Documents