set 1 - introduction & probability model

7/27/2019 Set 1 - Introduction & Probability Model

http://slidepdf.com/reader/full/set-1-introduction-probability-model 1/19

1

Dr. M. Hassan

ELE 360

Probability & Stochastic Processes

Introduction & Probability Model



2

Dr. M. Hassan

What is Probability?

• We are all aware of:– Deterministic events

• Daily sunrises and sunsets• Tides movement at the sea shore• Phases of the moon• Seasonal changes in weather

• Annual flooding of the Nile river … etc• What about:

– Randomevents• The outcome when you toss a coin or roll a die• Results of horse & car races• Variations of the stock market• Noise in communications systems strength of a received signal!• Waiting time in a traffic signal• Lifetime of a system• Waiting time at a cashier

Probability is the likelihood of an event happening (or not happening). Probabilitytypically tells us how certain we are about an event.



3

Dr. M. Hassan

Approaches to Assigning Probabilities

• Classical approach: make certain assumptions (such asequally likely, independence) about a situation

• Relative frequency approach: assign probabilities based onexperimentation or historical data

• Subjective approach: assign probabilities based on the

assignor’s judgment

• There are three ways to assign a probability, , to an outcome,, namely:



4

Dr. M. Hassan

Relative Frequency & Probability• Consider a random experiment the outcome of which is uncertain i.e. it

differs from a run to the other– Examples of such experiments would be a coin toss, dice rolling, ..etc

• Probability is a numerical measure of the likelihood of an event– It is a number that we attach to an event– A probability is a number from 0 to 1

• In general, the probability of an event can be approximated by therelative frequency , or the proportion of times that event occurs

The probability of an event is approximately =# of times the event occurs

# of experiments

As the probability is given by:

• If an outcome occurs times out of trials, its relative frequencyis and we define its probability to be approximately equal to

– “The probability of heads showing is ½ because when tossed repeatedly ,the coin will show heads half the time”



5

Dr. M. Hassan

Random Experiments & Trials• An random experiment is any act or process of observation that is when

performed leads to a single outcome that CANNOT be predicted withcertainty by the observer before the experiment– Theoretically, an experiment can be repeated (under the same conditions)

for an infinite number of times– The possible outcomes are defined by the observers according to their

interests– Each distinct outcome is called a “ sample point ” or a “basic outcome ”– A random experiment typically has a well-defined set of outcomes

• The experiment could be performed by a human, e.g. tossing a coin orrolling a die

• On the other hand, an outcome of an experiment could be just themeasurement of a naturally occurring random phenomenon, e.g. a noise

voltage– Atrial is one run of the experiment

The sample space of a random experiment is the set of all mutuallyexclusive and collectively exhaustive outcomes



6

Dr. M. Hassan

Probability Theory

• Consider for example the experiment of rolling a die:

• Sample space is the collection of all possible samplepoints (outcomes)

• Events are the results of experiments

• An event is a set of outcomes that meet certain criteria• An event with no sample points is called a null event

• An outcome or a sample point is an event that

cannot be decomposed into other events

The sample space is

An event could be “an even number is thrown”

An outcome

Events

• Random events are defined on a probability space that consists of asample space of all possible outcomes , a set of events that aresubsets of , and a probability measure which assigns a probability

to every event

What is ?



7

Dr. M. Hassan

Set Definitions• A set is a collection of objects (outcomes)

– Objects are called elements of the set– Outcomes are the elements of the sample space

• A set can be written as

• When an element belongs to a set , we write

• When an element does not belong to a set , we write

• A set can also be written asThis is called the rule method

This is called the tabular method

Tabularmethod isnot alwayspractical !!

• If all elements in a set areelements of the set , we say

that is a subset of ,denoted by:

• The subset relationship is atransitive relationship i.e.,



8

Dr. M. Hassan

Set Definitions (cont.)

• A set can be classified as:• Countable set

• The elements of which can be placed in one-to-one with the setof integers

• A set is said to be empty if it has no elements• An empty set is also called the null set and is referred to bythe symbol

• Uncountable set• A not countable set is uncountable• Example:

• Finite set• It is the set the elements of which are counted with a

terminating process• Infinite set

• If the set is not finite it is infinite. An infinite set could becountably infinite

Notice thedifference



9

Dr. M. Hassan

The Sample Space: Examples• Recall that the set of all possible outcomes of an experiment is calledthe sample space of the experiment

Examples• The experiment of a coin toss has the sample

space:

• The experiment of rolling a die has the samplespace:

These are examples ofdiscrete &finitesample space

• The experiment of measuring noise voltage has thesample space:

This is an example ofinfinite/continuous

sample space

• The experiment of randomly choosing a positive integerhas the sample space:

This is an exampleof countably infinite

&discrete samplespace



10

Dr. M. Hassan

Basic Set Operations• Set operations are better explained using Venn diagrams which areclosed-plane figures

The intersection of A and B

The union of A and BThe complement of A

The difference of A and B



11

Dr. M. Hassan

Set Algebra• Three theorems relate to laws of set operations including union &

intersection

• The distributive law

• The commutative law

• The associative law



12

Dr. M. Hassan

De Morgan’s Law & Duality Principle

• The De Morgan’s law:

• De Morgan’s law states that the complement of a union (intersection)of two sets and equals the intersection (union) of thecomplements and

• In application of De Morgan’s law to any expression, if we replaceunions by intersections and intersections by unions and each set byits complement, the identity is preserved

The duality principle:• The duality principle states that, for any identity, if we replace

unions by intersections and intersections by unions and the samplespace by the empty set and by the identity ispreserved

Example: Consider the identity given by

Applying the duality principle, we get



13

Dr. M. Hassan

What is an Event?• An event is a specific collection of sample points (outcomes) that meets

certain conditions– It is a subset of the sample space of an experiment– Events are defined on the sample space in a probabilistic way to select

subsets of the sample space– Events specify the interest in the characteristics of outcomes rather the

outcomes themselves– Two events are mutually exclusive when they have no common outcomes– All definitions & operations applicable to sets are also applicable to events

• The probability of an event is derived from the probabilities assigned tothe sample points associated with that event

and are events defined on the samplespace

– and are also events defined on

• Any subset of is called an eventExample:



14

Dr. M. Hassan

Example: Many Events Occur• In a roll of a die experiment, we define the events:

Events and have both occurredEvents and did not occur

The event has occurred

The event has also occurred … … and so on

• Assume now the observed outcome is 4, then:

Elementary or singleton events• An event that contains a single outcome is called an elementary event or

singleton event

– is an elementary event

- Also note that if 4 is a member of while is a subset of



15

Dr. M. Hassan

Outcomes versus Events (revisited)• Every trial results in only one outcome, say x is the outcome

• Fundamental notion: On each trial of the experiment, one outcome ( x inthis case) occurs, but many events may occur

• Of course, only the one elementary event { x} has occurred on this trial;all the other many events that have occurred have { x} as a subset, that

is, A occurs iff { x} AEvents Occur & Do Not Occur in Pairs

• Many (non-elementary) events may occur on this trial

One of the events A

and A c

always occurs

• Suppose that a trial resulted in outcome• Exactly one elementary event, occurred on this trial

• For any event , exactly one of the two events and occurred, andthe other did not

• Example: assume that the outcome of rolling a die is 4, then:– If the event occurred, this implies that the

event did not occur

– When occurs then the event willnot occur

16



16

Dr. M. Hassan

Two Special Events• can be regarded as a subset of• On any trial, the event always occurs

• The event is called the certain event or the sure event

• , the empty set, is also a subset of

• On any trial, the event never occurs

• The event is called the null event or the impossible event

17



17

Dr. M. Hassan

Disjoint Events• Events A and B are said to be disjoint or mutually exclusive if

• A and B have no elements in commonProbability of a disjoint union

• If events A and B are disjoint , then is said to be a disjoint unionof events

• For a disjoint union of events A and B ,• A consequence of this is the event A is the disjoint union of the elementary

events corresponding to its members

• If and are disjointevents

The probability of the union is given by:

18



18

Dr. M. Hassan

Axioms of Probability for Finite Spaces• The ideas described thus far are the basis of the axioms of probability

theory Probabilities are numbers assigned to events that satisfy thefollowing rules:

• Axiom I: for all events A

• Axiom II:• Axiom III: If events A and B are disjoint , then:

Consequences of the Axioms• and are disjoint events, then:

(Axiom III)

(Axiom II)

• But,

• Hence

19



19

Dr. M. Hassan

More Consequences of the Axioms

• and are disjoint events;

• Axiom I: for all events A

• Axiom II:• Axiom III: If events A and B are disjoint , then:

••

• Since (Axiom I), we deduce that:

set 1 - introduction & probability model

Documents