Measurement and Estimation of Network QoS among Peer Xbox Game Players

Youngki Lee, KAISTSharad Agarwal, Microsoft Research

Chris Butcher, Bungie StudioJitu Padhye, Microsoft Research

• A series of online multiplayer game via Xbox Live▫ First Person Shooter (FPS) game▫ 15 million copies sold worldwide

• We focus on Halo 3 for data collection and analysis.▫ Halo 3 has a large set of widely distributed player population.▫ released on September 25, 2007.

2

P2P architecture of Halo

3 Xbox console

P2P architecture of Halo

• P2P, a peer as a server

Xbox console

Xbox Live matchmaking service

• Network QoS between the server peer and other client peers is important for game quality.▫ excellent experience: latency (< 50ms), BW (50~70Kbps).▫ minimum requirement: latency (< 150ms), BW (>30Kbps).

QoS probing among peers

Xbox Live matchmaking service

Query: Give me a list of hosts that satisfy my criteria

Probing using the packet-pair technique

5

Candidate hosts

Motivation

• Understand network path quality (NPQ) among peer game players and characteristics of the players▫ NPQ in terms of network delay and capacity

• Address the problem of NPQ measurement overhead▫ improve user pre-game experience

probe fewer, better candidate hosts

• Limited publications on large-scale E2E network characteri-zation▫ Planetlab-based end-to-end NPQ studies: O(100) nodes▫ king-based end-to-end NPQ studies O(1000) nodes▫ several studies of provisioned server based games

6

Methodology

1. Collect probe data among peer game playersa) consoles report the probe results back to Xbox live service.

2. Understand characteristics of peer game playing

3. Understand NPQ between peer game players

4. Examine stability and predictability of NPQa) propose three simple predictors

IP history, prefix history, geography

b) examine robustness of the predictors

7

Outline• Background• Motivation• Analysis on probe data

▫ general characteristics▫ NPQ results

• NPQ prediction▫ IP history predictor▫ prefix history predictor▫ geography predictor

• Conclusion

8

Data

• Session data (per game attempted)▫ time, session-id, src IP

• NPQ measurement data (per probing to a host)▫ session-id, dest IP▫ # of packet-pairs sent, # of packet-pairs rcvd▫ minimum and median latency▫ average downstream and upstream capacity

• Player locations calculated from their IP addresses ▫ MaxMind database provides mapping between locations and

IP addresses

9

Basic statistics

10

•126 million probes among 5.6 million IP addresses !!!

11.14.2007 1.3.2008 (50 days)

sessions

distinct IPs

total probes

39,803,350

5,658,951

126,085,887

Geographic distribution

11

85% in USA

13% in Europe 2% in

Asia, Australia

Player characterization

• Strong diurnal pattern (peaks between 2 ~ 8PM, UTC time)

• Most players played a few games, only some a lot• Probe distribution per game trial (session)

▫ 90% of sessions probed fewer than 10 hosts, but some a lot.

1 10 100 10000.1

1

# of probes

cum

ula

tive

fre

q.

(sess

ion

s)

12

0.9

Delay distribution

• 25% of the delay measurement are above 150ms.▫ 150 ms: upper bound for responsive experience in FPS games.

13

1 10 100 1000 100000.01

0.1

1

delay (ms)

cum

ula

tive

fre

q.

(pro

bes)

150

0.75 25%

0 2000 4000 6000 8000 100000

1000000

2000000

3000000

4000000

5000000

6000000192Kbps

1.6Mbps5.8Mbps

10Mbps

capacity (Kbps)

freq

uen

cy (

x 1,0

00

,00

0)

(pro

bes)

Capacity distribution

• Peaks around typical broadband capacities in USA.▫ marginal error due to the packet pair technique.

14

Outline• Background• Motivation• Analysis on probe data

▫ general characteristics▫ NPQ results

• NPQ prediction▫ IP history predictor▫ prefix history predictor▫ geography predictor

• Conclusion

15

Predictors

• Predict NPQ without probing▫ to disqualify a host, select a host, do quick re-probe▫ potentially reduce the user-wait time and probe traffic

• IP/Prefix history predictor▫ reuse the previous probe results between the same IP pair▫ reuse results between two peers within the same prefix pair

determine prefixes by BGP table (12/27/2007 RouteViews)

• Geography predictor▫ predict delay or capacity based on the geographic distance

16

IP history predictor (delay)• Delays are very consistent over time, even for 50 days

▫ excellent predictor for delay

17

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mor

e0

0.2

0.4

0.6

0.8

1

Within 5 minWithin 30 minWithin 6 hrWithin 1 dayNo Constraints

coefficient of variation (CV)

cum

ulati

ve fr

eq.

(src

-dst

IP p

airs

)

• CV= Stdev/Mean, small CV = small variation

(50 days)

IP history predictor (capacity)• Capacities are also quite consistent over time.

▫ decent predictor for downstream capacity

18

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9 1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9 2

Mor

e0

0.2

0.4

0.6

0.8

1

Within 5 minWithin 30 minWithin 6 hrWithin 1 dayNo Constraints

coefficient of variation (CV)

cum

ulati

ve fr

eq.

(src

-dst

IP p

airs

)

(50 days)

Prefix history predictor• Quite consistent, but more variation compared to IP pairs

▫ outliers mostly caused the variation. ▫ good predictor for delay after removing outliers.

00.10.20.30.40.50.60.70.80.9

1

Within 5 minWithin 30 minWithin 6 hrWithin 1 dayNo Constraints

coefficient of variation (CV)

cum

ulati

ve f

req.

(s

rc-d

st p

refix

pai

rs)

19

(50 days)

Geography predictor

• Distance has strong correlation with minimum delay▫ good predictor for removing hosts with high latency

20

Distance (miles)

1200

1000

800

600

400

200

2000 4000 6000 8000 10000 120000

0

Dela

y (m

s)

Conclusions

• Large-scale end host latency and capacity characterization• Large-scale P2P game network characterization

▫ 126 million probes among 5.6 million unique IPs

• NPQ prediction for delay▫ IP history : great ! ▫ prefix history: good after removing outliers▫ geography : great for removing distant hosts

• NPQ prediction for capacity▫ IP history: decent!▫ prefix history: not feasible▫ geography: not feasible

21