Lecture Probability & statistics: Chapter 4

Lecture 4. BASIC PROBABILITY Lecture 4. BASIC PROBABILITY

 Probability  Outcome – Event  Complement Event, Intersection Event, Union Event  Mutually Exclusive, Independent, Collectively

Exhausive, Partitions

 Bernoulli Formula  Total Probability, Bayes’ Theorem

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 [1] Chapter 4, pp. 169 - 212

“Problem of Points” “Problem of Points”

 2 players A and B, contributed 50 Franc each.  The game is winner – loser only (no draw), A and B are

equally-likely to win in each match.

 Game rule: play 9 matches, the one wins more is final

winner and takes all of 100F

 But the game had stopped after 7 matches, and scores

at that time of A and B are 4 and 3, respectively.

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 How could they distribute the money?

4.1. Probability 4.1. Probability

 Probability is quantitative measure of uncertainty, the

chance that an uncertain event will occur.

 Probability: Subjective and Objective  Probability of event A is denoted by P(A)

 0 ≤ P(event) ≤ 1  P(always) = 1  P(impossible) = 0

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 () > (): A is more possible to occur than B

Experiment – Outcome - Event Experiment – Outcome - Event

Experiment Flip a coin Outcome Head, tail Event ‘Head’, ‘tail’

Toss a die 1,2,3,4,5,6 dot(s) ‘greater than 3’

Do an exam Score = 0, 1, 2,…, 10 ‘pass’;

Invest in a project Profit: (+), (–), zero

‘excellent’ ‘Non negative’ ‘profitable’

Apply for a job Pass, fail

PROBABILITY & STATISTICS– Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Do a job Salary = …

Classical Probability Classical Probability

 Sample space: all basic outcomes, denoted by Ω  Assumes all basic outcomes in the sample space are

equally-likely to occur

 Number of basic outcomes:  Number of basic outcomes for event A:  Probability of event A:

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Classical Probability Classical Probability

Example 4.1. Flipping a “fair” coin once, the probability that the Head is up = ?

Example 4.2. Flipping a fair coin two times. What is the probability of

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) “There are 2 Heads” (b) “There are 1 Head, 1 Tail” (c) “There are 2 Tails”

Classical Probability Classical Probability

Example 4.3: Flip a coin three times, what is the probability of

(a) There are 2 Heads only? (b) There are Heads only? (c) There are 2 Heads, given the first is Head? (d) There are 2 Heads, given the first is Tail?

Example 4.4 There is a box contains 6 white balls and 4 black balls. Random pick up 2 balls, calculate the probability of event that both balls are white, in 3 cases: (a) Pick up one, replace, then the next (b) Pick up one by one, without replacement (c) Pick up 2 balls simultaneously

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Example Example

 Frequency Table Probability Table

Male Female Sum

Freq.

160

240

400

Prob.

0.4

0.6

 Cross-Frequency Table Joint Probability Table

Freq. Single

Male Female Sum 140 80

Prob. Single

Male Female Sum 0.35 0.2 0.15

Married

100

160

260

Married 0.25

0.4

0.65 Sum 160

240

400 Sum 0.4

0.6

1

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

4.2. Probability vs Proportion 4.2. Probability vs Proportion

/  Number of experiments:  Frequency of event A:  Proportion of A:  When n is large enough: () ≈

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ex. In 100,000 new-borns, there were 51,000 boys, then the probability of “new-born is boy” is about 0.51 Ex. In 4,000 students, there are 600 fail in subject A, then probability of “Fail in subject A” is about 0.15

4.3. Complement Event 4.3. Complement Event

 Complement of A : all outcomes that not belong to A  Denoted by Ā

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ex.  A = “two flipped coins are Heads”  A = ?  B = “both picked balls are White”  B = ?  C = “all of students passed”  C = ?

Law of Complement Law of Complement

() = – ()  Law:

Ex. Flipping coin twice, what is the Probability of “Have at least one Tail” ?  Complement of “Have at least one tail” is “No any

Tail”, or “Two Heads”

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 Probability = 1 – P(Two Heads) = 1 – ¼ = ¾ 

4.4. Intersection Event 4.4. Intersection Event

 Intersection of A and B: all outcomes that belong to

AÇB

both A and B,  Denoted by ∩  ∩ ̅ = Ø

Ex. Pick 2 balls from the box of Blacks and Whites, A is “The first is white” , B is “The second is white”

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

AÇB is “Both balls are white”

Mutually Exclusive Mutually Exclusive

 A and B are Mutually exclusive : have no common  i.e., A Ç B = Ø

 A and Ā ?

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ex. Pick up 2 balls, A = “two Whites”, B = “two Blacks”, C = “At least one White”  A and B are mutually exclusive; B and C are mutually exclusive, but A and C are NOT mutually exclusive,

Conditional Probability Conditional Probability

 Conditional probability : probability of one event,

given that another event has occurred.

 The conditional probability of B given that A has

occurred, (B given A) denoted by P(B | A)

Ex. Flip a coin 3 times

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

P(2 Heads) = 3/8 = 0.375 P(2 Heads | The first is Head) = 2/4 = 0.5 P(2 Heads | The first is Tail) = 1/4 = 0.25 P(2 Heads | There are at least one Head) =

Conditional Probability Conditional Probability

Ex. Pick up one ball from box of 6 Whites and 4 Blacks, then one more. Let A = “the first ball is White”, B = “The second ball is White”

 Without replacement of the first ball: P(B | A) = 5/9  Replacement of the first ball: P(B | A) = 6/10

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ex. Pick up 3 balls one-by-one, without replacement  P(The third is White | Two firsts are Whites) =  P(The third is White | Two firsts are 1 White 1 Black) =  P(The third is White | Two firsts are Blacks) =

Independent Events Independent Events

 A and B are independent : A does not affect B, and B does

not affect A

( | ) = () and (|) = ()

 Û  A and B are not independent  dependent

Ex. From box of 6 Whites, 4 Blacks, pick up one by one Let A = “the 1st is White”, B = “The 2nd is White”.  Replace the first  A and B are independent

(|) = 6/10

(|) = (|Ā) = 6/10 ;

 Without replacement  A and B are dependent

(|) = 5/9

(|Ā) = 6/9

;

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Law of Intersection Law of Intersection

 ∩ = × = × (|)

 A and B are independent

Û ∩ = × ()

 Conditional Probability

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

∩

Law of Intersection Law of Intersection

Ex. Pick one ball from box of 6 whites and 4 blacks, then one more. Let A = “the first ball is White”, B = “The second ball is White”.  The Probability of “Two White” = (Ç)

(Ç) = () × (|)

 Without replacement

Ç = × =

 Replacement

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ç = × =

Example Example

Example 4.5. The chance that a student passes the subjects A and B are 0.6 and 0.8, respectively. However, if passed subject A, the chance for him to pass subject B is 0.9

(a) What is probability that he passes both subjects? (b) Whether “Pass subject A” and “Pass subject B” are

independent?

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

been passed?

In General In General

 Intersection of n events A1 and A2 and… and An

∩ ∩ ⋯ ∩

 A1, A2,…,An are totally independent Û

( ∩ ∩ ⋯ ∩ ) = () × ⋯ × ()

Ex. Pick 5 balls from box of 6 Whites and 4 Backs, with replacement, the Probability that all of them are white is

× × × × = 0.6

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

6 10 6 10 6 10 6 10 6 10

4.5. Union Events 4.5. Union Events

 Union of A and B: all outcomes belong to either A or B  Denoted by ∪  ∪ includes:

∩ ; ∩ ; ̅ ∩

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Ex. Pick 2 balls from box of 6Ws, 4Bs A = “the first is W”, B = “the second is W”,  then AB is “at least one W”

Law of Union Law of Union

∪ = + − ( ∩ )

 A and B are mutually exclusive events

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Û ∩ = Ø Û ( ∩ ) = 0 Û ( ∪ ) = () + ()

Example Example

Example 4.6. The chance that a candidate passes the subjects A and B are 0.6 and 0.8, respectively. However, if passed subject A, the chance for him to pass subject B is 0.9. What is probability that (s)he:

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) pass both subjects? (b) pass at least one subject? (c) fail in both subjects? (d) fail in at least one subject?

4.6. Joint Probability 4.6. Joint Probability

Subject B

Sum

Join Probability Table

Pass B

Pass A

Ex. The chance that a candidate passes the subjects A and B are 0.6 and 0.8, respectively. The chance of passing both is 0.54. Building the join probability table

0.54 Fail B 0.06

0.6

Sub. A

0.26

0.14

0.4

Fail A Sum

0.8

0.2

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Joint Probability Joint Probability

Marginal prob. distributions

Subject B

Sum

Join Probability Table

Pass B

0.6 A 0.4

Pass A

Fail B 0.54 0.06

0.6

Sub. A

Fail A

0.26 0.14

0.4

Sum

0.8 B 0.2

0.8

0.2

1

Example 4.7. Base on the Join Probability table, build the Marginal probability distribution, and what is the probability of

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) Pass B given passed A? (b) Pass B given failed A? (c) Fail A given passed B?

Example Example

Example 4.8. The exam includes two independent

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

questions. The probability that student’s answer is correct in the A and B question are 0.7 and 0.5. What is the probability that a student (a) Correct in at least one answer? (b) Correct in only one answer? (c) Incorrect in at least one answer? (d) Incorrect in both answers? (e) Joint probability table?

Problem Problem

Sum

Join prob. of Customers

Male: (M)

0.45

Female: (F)

0.55

Sum

Buy goods only (BO) 0.2 0.15 0.35

Buy goods and use services (BS) 0.15 0.3 0.45

Use Services only (SO) 0.1 0.1 0.2 Problem 4.9. Base on the table, fill the blanks, and build the marginal prob. tables; what is the proportion of:

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) the customer who use services (b) good only buyers in female customers (c) services users only in male customers (d) male in customers who buy goods only (e) female in customers who use services

In General In General

 Union of A1 or A2 or… or An is denoted by

A1  A2  …  An

 A1, A2,…,An are totally mutually exclusive  P(A1A2…An) = P(A1) + P(A2) +…+ P(An)

Example 4.10. There are 3 optional subjects 1, 2, 3. Each of two students B and C randomly chooses one subject, assumed that they are independently. What the probability that they choose the same subject?

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Collectively Exhaustive & Partitions Collectively Exhaustive & Partitions

 A1, A2,..., An are Collectively Exhaustive:

A1A2 ... An = Ω

 A1,A2,...,An are Collectively Exhaustive and Mutually

... An

A1 A2 A3

Ω

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Exclusively  They are Partitions  A and Ā are Partitions

 A1, A2,..., An are Partitons : P(A1) + P(A2) +…+ P(An) = 1

Ex. Flip coin twice; A1 is (2 Heads), A2 is (2 Tails), A3 is (1 Head 1Tail), A4 is (At least one Head).  Whether the following groups are Collectively

Exhaustive, Partitions, Complement?

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) A1,A2,A3 (b) A1,A2,A4 (c) A1,A3,A4 (d) A2, A4

4.7. Bernoulli Formula 4.7. Bernoulli Formula

Problem 4.11. An quiz has 3 independent questions, the chance for a candidate to answer correctly each question are equally, and be 0.6. What is the probability that a candidate will be

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

(a) Correct in only one answer? (b) Correct in only two answers? (c) Correct in at least one answer?

Bernoulli Formula Bernoulli Formula

 There are independent experiments  The probability that event A occurs in each

experiment is equally and denoted by , then probability of A do not occur is – .  Bernoulli Trial, denoted by B(n, p)

 The probability that in n experiments, event A occurs

−

in times:

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

, =

Example Example

Example 4.12. The test includes 10 independent multiple choice questions. Each question has 4 options with only one correct choice.  A candidate randomly answers all question. What is

the probability that (s)he:

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 (a) Correct in 4 answers  (b) Correct in at least one answer

3.8. Total Probability – Bayes’ Theorem 3.8. Total Probability – Bayes’ Theorem

Example 4.13. There are 2 boxes, Box type I contains 6 white balls and 4 black ones, Box type II contains 8 white balls and 2 black ones.

(a) Random choose one box, then random choose one ball. What is the probability that the ball is white? (b) If the chosen ball is white, what is the probability that

the chosen box is Type I, Type II?

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

the chosen box is Type I, Type II ?

Problem Problem

 (White ball) =

= (Box I and White ball) or (Box II and White ball) = (B1ÇW)  (B2 ÇW)

= 0.3 + 0.4 = 0.7

 = × × +

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 Probability of 0.7 is contributed by 0.3 from Box I and 0.4 from Box II. Therefore Probability that given Ball is white, chosen box is Box type I =

Example Example

 Use table to calculate and analyze Total probability

and contribution probability

() (|) (Ç) (|) 0.3 / 0.7

0.5

0.3

0.6

0.8

0.5

0.4 0.4 / 0.7

Box I (B1) Box II (B2) Sum

1

0.7

1

 Example: What happens if there are 3 boxes of type I,

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

and 2 boxes of type II ?

Bayes’ Theorem Bayes’ Theorem

 B1,B2,...,Bk are partitions  Total probability:

() = ()(|) + ⋯ + ()(|)

= ∑ ()(|)

 Bayes’ Theorem: Probability of Bi occurs, given A

occurred is P(Bi|A)

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

= (|) ()

Bayes’ Theorem Bayes’ Theorem

 () :  (|) : prior-probability posterior-probability

2nd Stage P(A|Bi) ( |) ( |) ...

()

( |)

1st Stage Prior: P(Bi) B1 () B2 () ... ... Bk Sum

P(AÇBi) ( ∩ ) ( ∩ ) ... ( ∩ ) P(A)

Posterior P(Bi|A) ( ∩ ) / () ( ∩ ) / () ... ( ∩ ) / () 1

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

Key concepts Key concepts

PROBABILITY & STATISTICS – Bui Duong Hai – NEU – www.mfe.edu.vn/buiduonghai

 Events, Probability, Conditional probability  Complement, Partitions, Intersection, Union  Independent, Mutually exclusive  Probability Rules, Bernoulli formula  Bayes’ Theorem