week 1: introductionstatweek 1: introduction contents organization of course. motivation for...

http://statwww.epfl.ch

Week 1: Introduction

Contents

Organization of course.

Motivation for probability and statistics.

Basic notions of sets and of combinatorics.

References: Ross (Chapter 1); Ben Arous notes (Chapter 1).

Exercises: 1–16 of Recueil d’exercices.

Probability and Statistics I — week 1 1


Organization

Lecturer: Professor A. C. Davison

Assistants: D. Baraka, S. Brahim Belhaouar, S. Salom

Lectures: Monday 14.15–16.00, CO1

Exercises: Monday 16.15–18.00, CO1, CO5. Students should come

in alternate weeks. Those whose surnames begin with letters A–L

should come in odd weeks (starting today), and those whose

surnames begin with letters M–Z should come in even weeks (starting

next week).

Tests: December 15 2003, March 29 2004, June 14 2004.



Course Material

Books: Roughly the first two-thirds of the course is on probability,

and a good account is:

Ross, S. M. (1999) Initiation aux Probabilites. PPUR: Lausanne.

For notes in French by Professor Gerard Ben Arous, see

http://dmawww.epfl.ch/benarous/Pmmi/prost1/prost1_fr00.htm

There are many other excellent introductory books: look in the

library.

References on statistics will be given later.

Exercises: We use the Recueil d’Exercices, which is also available

electronically:

http://ima.epfl.ch/cours/cours.htm



Intellectual Motivation

Probability and statistics provide mathematical tools and models for

studying random events:

• lotteries, weather forecasting, finance (Nobel Prize, 2003), . . .;

• numbers of junk emails I receive today;

• burstiness of internet traffic;

• noise affecting transmission of a signal or an image;

• errors in coding of signals.

They provide optimal methods for forecasting, for filtering out the

noise, for suggesting how traffic should be handled, and for

reconstruction of the true signal or image.



The scum of the universe

Log gamma ray counts indexed by galactic latitude and longitude.

0

1

2

3

4

5

6EGRET measurements

50100150200250300350

20

40

60

80

100

120

140

160

180



Markov random field

c

c

c

c

c

c

c

c

c

c

c

c

c

c

c

c

c c

c

c



One-dimensional slice

50 100 1500

5

10

15

20

25

EGRET measurements

50 100 1500

5

10

15

20

25

Wavelet−Anscombe estimate

50 100 1500

5

10

15

20

25

Wavelet estimate

50 100 1500

5

10

15

20

25

MRF−l1 estimate



A better view?

EGRET measurements

100200300

50

100

150

Wavelet−Anscombe estimate

100200300

50

100

150

Wavelet estimate

100200300

50

100

150

MRF−l1 estimate

100200300

50

100

150



Practical Motivation

Many subsequent SSC courses use probability ideas:

Learning and neural networks (Hasler/Thiran);

Performance evaluation (Le Boudec);

Statistical signal processing and applications (Vetterli);

Automatic speech processing (Bourlard);

Biomedical signal processing (Vesin);

Stochastic models for communications (Thiran);

Signal processing for communications (Prandoni);

Information theory and coding (Teletar);

. . .



Notes and Plan

The transparencies will be available about a week before each lecture,

and should be downloaded, printed two-to-a-side, and brought to the

lecture. See

http://statwww.epfl.ch/davison/teaching/ProbStatSC/20032004/

I will bring NO copies of the transparencies to the lectures.

Week 1: preliminaries on sets and counting

Week 2: basic notions of probability spaces



Preliminaries on Sets

Definition: A set A is a collection of objects, x1, x2, . . . , xn, . . .:

A = x1, x2, . . . , xn, . . . .

We write x ∈ A to mean ‘x is an element of A’, or ‘x belongs to A’.

The collection of all possible objects in a given context is called the

universal set Ω.

Examples of sets are

CH = Geneve, Vaud, . . . , Grisons set of Swiss cantons

0, 1 = finite set consisting of elements 0 and 1

N = natural numbers, countable set

R = real numbers, uncountable set

∅ = empty set, has no elements



Subsets

Definition: A set A is a subset of a set B if x ∈ A implies that

x ∈ B: we write A ⊂ B.

If A ⊂ B and B ⊂ A, then every element of A is contained in B and

vice versa, so A = B: both sets contain precisely the same elements.

Notice that ∅ ⊂ A for any set A.

Thus for example:

∅ ⊂ 1, 2, 3 ⊂ N ⊂ Z ⊂ Q ⊂ R ⊂ C, I ⊂ C

Venn diagrams are useful for understanding elementary set

relations, but beware: they can be misleading (not every relation can

be so represented).



Cardinal of a Set

Definition: A finite set A has a finite number of elements, and this

is called its cardinal(ity):

card A, #A, |A|.

Obviously |∅| = 0 and |0, 1, | = 2, but | R | does not exist (at least

for this course!)

Exercise: Show that if A and B are finite and A ⊂ B, then

|A| ≤ |B|.



Boolean Operations

Definition: Let A, B ⊂ Ω. Then we define three operations:

the union of A and B is A ∪ B = x ∈ Ω : x ∈ A or x ∈ B;

the intersection of A and B is A ∩ B = x ∈ Ω : x ∈ A and x ∈ B;

and the complement of A in Ω is Ac = x ∈ Ω : x 6∈ A.

Obviously A ∩ B ⊂ A ∪ B, and if the sets are finite, then

|A| + |B| = |A ∩ B| + |A ∪ B|, |A| + |Ac| = |Ω|.

We can also define the difference of A and B to be

A \ B = A ∩ Bc = x ∈ Ω : x ∈ A and x 6∈ B,

(note that A \ B 6= B \ A), and the symmetric difference

A 4 B = (A \ B) ∪ (B \ A).



Boolean Operations

If Aj∞

j=1is an infinite set of subsets of Ω, then

∞⋃

j=1

Aj = A1 ∪ A2 ∪ A3 ∪ · · · : all x ∈ Ω in at least one Aj

∞⋂

j=1

Aj = A1 ∩ A2 ∩ A3 ∩ · · · : all x ∈ Ω in every Aj

The following are easy to show (mostly using Venn diagrams):

• (Ac)c = A, (A ∪ B)c = Ac ∩ Bc, (A ∩ B)c = Ac ∪ Bc

• A∩ (B∪C) = (A∩B)∪ (A∩C), A∪ (B∩C) = (A∪B)∩ (A∪C)

• (⋃

∞

j=1Aj)

c =⋂

∞

j=1Ac

j , (⋂

∞

j=1Aj)

c =⋃

∞

j=1Ac

j



Partition

Definition: A partition of Ω is a collection of non-empty subsets

A1, . . . , An of Ω such that

1. the Aj are exhaustive, that is, A1 ∪ A2 ∪ · · · ∪ An = Ω, and

2. the Aj are disjoint, that is, Ai ∩ Aj = ∅, whenever i 6= j.

A partition can also have an infinite number of sets Aj∞

j=1.

Example 1.1: Let Aj = [j, j + 1), for j = . . . ,−1, 0, 1, . . .. Do the

Aj partition Ω = R?

Example 1.2: Let Aj be the set of all natural numbers divisible by

j, for j = 1, 2, . . .. Do the Aj partition Ω = N?



Cartesian Product

Definition: The Cartesian product of two sets A, B is the set of

ordered pairs

A × B = (a, b) : a ∈ A, b ∈ B.

Likewise

A1 × · · · × An = (a1, . . . , an) : a1 ∈ A1, . . . , an ∈ An.

If A1 = · · · = An = A, then we write A1 × · · · × An = An.

As the pairs are ordered, A × B 6= B × A unless A = B.

If A1, . . . , An are all finite, then

|A1 × · · · × An| = |A1| × · · · × |An|.

Example 1.3: Let A = a, b, B = 1, 2, 3. Write down A × B.



Preliminaries on Combinatorics

Combinatorics is the mathematics of counting. Two basic principles:

• addition: if I have m red hats and n blue hats, then I have

m + n hats in total;

• multiplication: if I have m hats and n scarves there are mn

different ways I can combine them.

In mathematical terms, let A1, . . . , Ak be sets. Then

|A1 × · · · × Ak| = |A1| × · · · × |Ak|,

and if the Aj are disjoint, then

|A1 ∪ · · · ∪ Ak| = |A1| + · · · + |Ak|.



Examples

Example 1.4: Six dice are rolled.

(a) How many outcomes are there?

(b) For how many of these do the dice show six different faces?

Example 1.5: I have 5 hats and 5 friends, and I want to give one

hat to each friend. How many ways can I do this?

Example 1.6: I have 5 hats and 3 friends, and I want to give one

hat to each friend. How many ways can I do this?

Example 1.7: How many different ways can I arrange my 4

probability books on a shelf?



Permutations: Ordered Selection

Definition: A permutation of n distinct symbols is an ordering of

them.

Theorem: Given n distinct symbols, the number of distinct

permutations (without repetition) of length r ≤ n is

n (n − 1) (n − 2) · · · (n − r − 1) =n!

(n − r)!.

Thus there are n! permutations of length n.

Theorem: Given n =∑r

i=1ni symbols of r distinct types, where ni

are of type i and are otherwise indistinguishable, the number of

permutations (without repetition) of all n symbols is

n!

n1! n2! · · · nr!.



Example

Example 1.8: You are playing bridge, and when you pick up your

cards you notice that the suits are already grouped: the clubs are all

adjacent to each other, the hearts likewise, and so on. Your hand

contains 4 spades, 4 hearts, 3 diamonds and 2 clubs.

(a) How many permutations of your hand are there?

(b) How many of these permutations have the cards grouped together

as described?



Example

Example 1.9: A class of 20 students elects a committee of size 4 to

organise a voyage detudes. How many ways are there of choosing the

committee if:

(a) their roles are indistinguishable?

(b) there are 4 distinct roles (president, secretary, treasurer,

travel-agent)?

(c) there is a president, a treasurer, and two travel-agents?

(d) there are two treasurers and two travel-agents?



Multinomial and Binomial Coefficients

Definition: Let n1, . . . , nr lie in the range 0, 1, . . . , n, with total

n1 + · · · + nr = n. Then(

n

n1, n2, . . . , nr

)

=n!

n1! n2! · · · nr!,

is called a multinomial coefficient. The case r = 2 is most

common:(

n

k

)

=n!

k!(n − k)!

(

= Ckn in some older books

)

is called a binomial coefficient.

Example 1.10: Compute these coefficients for n = 1, 2, 3, 4.



Combinations: Unordered Selection

Theorem: The number of ways of choosing a set of r symbols from a

set of n distinct symbols without repetition is

n!

r!(n − r)!=

(

n

r

)

.

Theorem: The number of ways of dividing n distinct objects into r

distinct groups of sizes n1, . . . , nr, where n1 + · · · + nr = n is

n!

n1! n2! · · · nr!.



Properties of Binomial Coefficients

Theorem: If n, m are non-negative integers and r ∈ 0, . . . , n, then:(

n

r

)

=

(

n

n − r

)

;

(

n + 1

r

)

=

(

n

r − 1

)

+

(

n

r

)

, (Pascal’s triangle);

r∑

j=0

(

m

j

)(

n

r − j

)

=

(

m + n

r

)

, (Vandermonde’s formula);

(a + b)n =n

∑

r=0

(

n

r

)

arbn−r, (Newton’s binomial formula);

(1 − x)−n =

∞∑

j=0

(

n + j − 1

j

)

xj , |x| < 1 (negative binomial series).



Partitions of Integers

Example 1.11: How many ways are there of putting 6 identical

balls into 3 boxes, so that each box contains at least one ball?

Example 1.12: How many ways are there of putting 6 identical

balls into 3 boxes?

Theorem: (a) The number of distinct vectors (n1, . . . , nr) of positive

integers satisfying n1 + · · · + nr = n is(

n − 1

r − 1

)

.

(b) The number of distinct vectors (n1, . . . , nr) of non-negative

integers satisfying n1 + · · · + nr = n is(

n + r − 1

n

)

.


week 1: introductionstatweek 1: introduction contents organization of course. motivation for...

Documents