modeling social data, lecture 2: introduction to counting

Introduction to CountingAPAM E4990

Modeling Social Data

Jake Hofman

Columbia University

January 30, 2013

Jake Hofman (Columbia University) Intro to Counting January 30, 2013 1 / 28

Why counting?


Why counting?

sta·tis·tic

noun1. A function of a random sample of data


Why counting?

p( y︸︷︷︸support

| x︸︷︷︸age

)


Why counting?

?p( y︸︷︷︸

support

| x1, x2, x3, . . .︸︷︷︸age, sex, race, party

)


Why counting?

Problem:

Traditionally difficult to obtain reliable estimates due to smallsample sizes or sparsity

(e.g., ∼ 100 age × 2 sex × 5 race × 3 party = 3000 groups,but typical surveys collect ∼ 1,000s of responses)


Why counting?

Potential solution:

Sacrifice granularity for precision, by binning observations intolarger, but fewer, groups

(e.g., bin age into a few groups: 18-29, 30-49, 50-64, 65+)


Why counting?

Potential solution:

Develop more sophisticated methods that generalize well fromsmall samples

(e.g., fit a model: support ∼ β0 + β1age + β2age2 + . . .)


Why counting?

(Partial) solution:

Obtain larger samples through other means, so we can just countand divide to make estimates via relative frequencies

(e.g., with ∼ 1M responses, we have 100s per group and canestimate support within a few percentage points)


Why counting?

The good:

Shift away from sophisticated statistical methods on small samplesto simple methods on large samples


Why counting?

The bad:

Even simple methods (e.g., counting) are computationallychallenging at large scales

(1M is easy, 1B a bit less so, 1T gets interesting)


Why counting?

Claim:

Solving the counting problem at scale enables you to investigatemany interesting questions in the social sciences


Learning to count

This week:

Counting at small/medium scales on a single machine

Following weeks:

Counting at large scales in parallel


Counting, the easy way

Split / Apply / Combine1

http://bit.ly/sactutorial

• Load dataset into memory

• Split: Arrange observations into groups of interest

• Apply: Compute distributions and statistics within each group

• Combine: Collect results across groups1http://bit.ly/splitapplycombine


http://bit.ly/sactutorial

http://bit.ly/splitapplycombine

The generic group-by operation

Split / Apply / Combine

for each observation as (group, value):

place value in bucket for corresponding group

for each group:

apply a function over values in bucket

output group and result

Useful for computing arbitrary within-group statistics when wehave required memory

(e.g., conditional distribution, median, etc.)


Why counting?


Example: Anatomy of the long tail

Dataset Users Items Rating levels ObservationsMovielens 100K 10K 10 10M

Netflix 500K 20K 5 100M


http://www.grouplens.org/node/73

Example: Movielens

0

1,000,000

2,000,000

3,000,000

1 2 3 4 5Rating

Num

ber

of r

atin

gs


Example: Movielens

0

1,000,000

2,000,000

3,000,000

1 2 3 4 5Rating

Num

ber

of r

atin

gs

group by rating value

for each group:

count # ratings


Example: Movielens

1 2 3 4 5Mean Rating by Movie

Den

sity


Example: Movielens

group by movie id

for each group:

compute average rating


Den

sity


Example: Movielens

0%

25%

50%

75%

100%

0 3,000 6,000 9,000Movie Rank

CD

F


Example: Movielens

0%

25%

50%

75%

100%

0 3,000 6,000 9,000Movie Rank

CD

F

group by movie id

for each group:

count # ratings

sort by group size

cumulatively sum group sizes


Example: Movielens

0

2,000

4,000

6,000

8,000

100 10,000User eccentricity

Num

ber

of u

sers


Example: Movielens

join movie ranks to ratings

group by user id

for each group:

compute median movie rank

0

2,000

4,000

6,000

8,000

100 10,000User eccentricity

Num

ber

of u

sers





What do we do when the full dataset exceeds available memory?







Sampling?Unreliable estimates for rare groups







Random access from disk?1000x more storage, but 1000x slower2

2Numbers every programmer should knowJake Hofman (Columbia University) Intro to Counting January 30, 2013 21 / 28


https://gist.github.com/jboner/2841832





StreamingRead data one observation at a time, storing only needed state



The combinable group-by operation

Streaming


if new group:

initialize result

update result for corresponding group as function of

existing result and current value

for each group:


Useful for computing a subset of within-group statistics with alimited memory footprint

(e.g., min, mean, max, variance, etc.)


Example: Movielens

0

1,000,000

2,000,000

3,000,000

1 2 3 4 5Rating

Num

ber

of r

atin

gs

for each rating:

counts[movie id]++


Example: Movielens

for each rating:

totals[movie id] += rating

counts[movie id]++

for each group:

totals[movie id] /

counts[movie id]


Den

sity


Yet another group-by operation

Per-group histograms


histogram[group][value]++

for each group:

compute result as a function of histogram


We can recover arbitrary statistics if we can afford to store countsof all distinct values within in each group


The group-by operation

For arbitrary input data:

Memory Scenario Distributions StatisticsN Small dataset Yes General

V*G Small distributions Yes GeneralG Small # groups No CombinableV Small # outcomes No No1 Large # both No No

N = total number of observationsG = number of distinct groups

V = largest number of distinct values within group


Examples (w/ 8GB RAM)

Median rating by movie for Netflix

N ∼ 100M ratingsG ∼ 20K movies

V ∼ 10 half-star values

V *G ∼ 200K, store per-group histograms for arbitrary statistics

(scales to arbitrary N, if you’re patient)



Median rating by video for YouTube

N ∼ 10B ratingsG ∼ 1B videos


V *G ∼ 10B, fails because per-group histograms are too large tostore in memory

G ∼ 1B, but no (exact) calculation for streaming median



Mean rating by video for YouTube

N ∼ 10B ratingsG ∼ 1B videos


G ∼ 1B, use streaming to compute combinable statistics


The group-by operation

For pre-grouped input data:

Memory Scenario Distributions StatisticsN Small dataset Yes General

V*G Small distributions Yes GeneralG Small # groups No CombinableV Small # outcomes Yes General1 Large # both No Combinable

N = total number of observationsG = number of distinct groups

V = largest number of distinct values within group


modeling social data, lecture 2: introduction to counting

Science

counting problem

smallmedium scales

sophisticated methods

generic group

age 2age2

bin age

potential solution

small samplesto simple