Chapter 4: Probability Theory and Distributions - Summary and Exercises
md8 min readSummary of chapter 4
Summary
1. Sample Spaces and Events
Sample Space (): The complete set of all possible outcomes of a random experiment.
Examples:
- Coin flip:
- Die roll:
- Temperature:
Events: Subsets of the sample space. An event A occurs if the outcome lies within that subset.
Set Operations:
- Union (): At least one of A or B occurs
- Intersection (): Both A and B occur
- Complement (): A does not occur
- De Morgan's Laws: and
2. Conditional Probability and Independence
Conditional Probability: The probability of event A given that B has occurred.
Independence: Events A and B are independent if:
3. Bayes' Theorem
The fundamental rule for updating probabilities based on new evidence:
Where:
- P(B): Prior probability
- P(A | B): Likelihood
- P(A): Evidence (total probability)
- P(B | A): Posterior probability
4. Random Variables
Definition: A function that assigns a real number to each outcome in the sample space:
Types:
- Discrete: Takes countable values (e.g., dice rolls, coin flips)
- Continuous: Takes any value in an interval (e.g., temperature, height)
5. Expected Value and Variance
Expected Value (Mean):
- Discrete:
- Continuous:
Variance:
Standard Deviation:
6. Key Probability Distributions
Bernoulli Distribution
Models a single binary trial (success/failure).
- PMF:
- Mean:
- Variance:
Binomial Distribution
Models the number of successes in n independent Bernoulli trials.
- PMF:
- Mean:
- Variance:
Poisson Distribution
Models the number of events occurring in a fixed interval.
- PMF:
- Mean:
- Variance:
Normal (Gaussian) Distribution
The bell curve distribution, fundamental in statistics.
- PDF:
- Mean:
- Variance:
Standard Normal:
Z-score transformation:
Written Exercises
Exercise 1: Sample Spaces and Events
A fair six-sided die is rolled once.
Questions:
- Define the sample space .
- Define the following events:
- A: Rolling an even number
- B: Rolling a number greater than 4
- C: Rolling a prime number
- Find:
Solution
-
Sample Space:
-
Events:
- A = {2, 4, 6} (even numbers)
- B = {5, 6} (greater than 4)
- C = {2, 3, 5} (prime numbers)
-
Set Operations:
(even OR greater than 4):
(even AND greater than 4):
(NOT even, i.e., odd):
(greater than 4 BUT NOT even):
Exercise 2: Conditional Probability
A company produces laptops. From historical data:
- 5% of laptops are defective:
- 95% of laptops are non-defective:
- A quality test correctly identifies 90% of defective laptops:
- The test incorrectly flags 8% of non-defective laptops as defective:
Questions:
- What is the probability that a randomly selected laptop tests positive, ?
- If a laptop tests positive, what is the probability it is actually defective, ?
Solution
-
using the law of total probability:
-
using Bayes' Theorem:
Interpretation: Even though the test is 90% accurate for defective laptops, only about 37.2% of laptops that test positive are actually defective. This is because defective laptops are rare (only 5%), so false positives dominate.
Exercise 3: Expected Value and Variance
A discrete random variable X has the following probability distribution:
| X | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| P(X) | 0.1 | 0.3 | 0.4 | 0.2 |
Questions:
- Verify that this is a valid probability distribution.
- Calculate .
- Calculate .
- Calculate .
- Calculate the standard deviation .
Solution
-
Validity Check:
-
Expected Value:
-
:
-
Variance:
-
Standard Deviation ():
Exercise 4: Binomial Distribution
A student takes a multiple-choice exam with 10 questions. Each question has 4 choices, and the student guesses randomly on every question.
Questions:
- What is the probability of success (correct answer) on a single question?
- Let = number of correct answers. What distribution does follow?
- What is the probability the student gets exactly 3 questions correct?
- What is the expected number of correct answers?
- What is the variance of the number of correct answers?
Solution
-
Probability of Success:
-
Distribution:
-
:
Calculate binomial coefficient:
Calculate probability:
-
Expected Value:
-
Variance:
Exercise 5: Poisson Distribution
A call center receives an average of 4 calls per hour.
Questions:
- What distribution models the number of calls in one hour?
- What is the probability of receiving exactly 5 calls in one hour?
- What is the probability of receiving at most 2 calls in one hour?
- What is the expected number of calls and the variance?
Solution
-
Distribution:
-
:
-
:
-
Expected Value and Variance:
For Poisson distribution:
Exercise 6: Normal Distribution and Z-scores
The heights of adult men in a population follow a normal distribution with mean cm and standard deviation cm.
Questions:
- What is the Z-score for a man who is 183 cm tall?
- What is the Z-score for a man who is 160 cm tall?
- Using the standard normal table (or knowing that ), what proportion of men are taller than 183 cm?
- What height corresponds to the 95th percentile? (Use )
Solution
-
Z-score for 183 cm:
-
Z-score for 160 cm:
-
Proportion taller than 183 cm:
We need
About 15.87% of men are taller than 183 cm.
-
95th Percentile Height:
We need such that
This corresponds to Z = 1.645
The 95th percentile height is approximately 188 cm.
Exercise 7: Bayes' Theorem - Medical Diagnosis
A disease affects 1% of the population. A diagnostic test has the following properties:
- Sensitivity (true positive rate):
- Specificity:
Questions:
- What is the false positive rate ?
- What is the probability a randomly selected person tests positive?
- If a person tests positive, what is the probability they actually have the disease?
Solution
-
False Positive Rate:
-
using total probability:
-
using Bayes' Theorem:
Interpretation: Even with a 95% sensitive test, only about 8.76% of people who test positive actually have the disease. This is because the disease is rare (1%), so false positives outnumber true positives.
Python Code Examples
Example 1: Binomial Probability Calculation
import scipy.stats as stats
# Parameters
n = 10
p = 0.25
k = 3
# Calculate P(X = k)
prob = stats.binom.pmf(k, n, p)
print(f"P(X = {k}) = {prob:.4f}")
# Expected value and variance
mean = n * p
variance = n * p * (1 - p)
print(f"E[X] = {mean}")
print(f"Var(X) = {variance}")Example 2: Normal Distribution Calculations
import scipy.stats as stats
# Parameters
mu = 175
sigma = 8
x = 183
# Calculate Z-score
z = (x - mu) / sigma
print(f"Z-score for {x} cm: {z:.2f}")
# Calculate probability
prob_less = stats.norm.cdf(z)
prob_greater = 1 - prob_less
print(f"P(X > {x}) = {prob_greater:.4f}")
# Find 95th percentile
percentile_95 = stats.norm.ppf(0.95, loc=mu, scale=sigma)
print(f"95th percentile: {percentile_95:.2f} cm")Example 3: Poisson Probability
import scipy.stats as stats
# Parameter
lambda_val = 4
# P(X = 5)
prob_5 = stats.poisson.pmf(5, lambda_val)
print(f"P(X = 5) = {prob_5:.4f}")
# P(X <= 2)
prob_le_2 = stats.poisson.cdf(2, lambda_val)
print(f"P(X <= 2) = {prob_le_2:.4f}")