Chap 4- Summary

md3 min readSummary of chapter 4

  • Sample Space: defining all possible outcomes / can be discrete → number of values is finite, or continuous → infinitely many values in an interval

  • Event: subset of sample space / specific situation. If outcome is in a subset we say event has occurred

    • certain event → always occurs (Ω\Omega)
    • impossible event → never occurs (∅\emptyset)
  • Combining events:

    • Union (A∪BA \cup B): event that either A or B or both occur
    • Intersection (A∩BA \cap B): event that both A and B occur
    • Complement (AcA^c): event that A does not occur
    • Difference (A−BA - B): event that A occurs but B does not occur
  • De Morgan's Laws:

    • (A∪B)c=Ac∩Bc(A \cup B)^c = A^c \cap B^c
    • (A∩B)c=Ac∪Bc(A \cap B)^c = A^c \cup B^c
  • Conditional Probability of A given B:

    • P(A∣B)=P(A∩B)P(B)P(A|B) = \frac{P(A \cap B)}{P(B)}, for P(B)>0P(B) > 0
  • Independence of events: meaning knowing one gives us no information about the other

    • P(A∣B)=P(A)P(A|B) = P(A)
    • P(B∣A)=P(B)P(B|A) = P(B)
    • P(A∩B)=P(A)P(B)P(A \cap B) = P(A)P(B)
  • Bayes' Theorem: B is true once A is observed

    • P(B∣A)=P(A∣B)P(B)P(A)P(B|A) = \frac{P(A|B)P(B)}{P(A)} for P(A)>0P(A) > 0
    • with:
      • P(B)P(B) prior probability (initial belief in event B before seeing any data)
      • P(A∣B)P(A|B) likelihood (probability of observed data A, given B being true)
      • P(B∣A)P(B|A) posterior probability (updated belief in event B after seeing data A)
  • Random Variable: a function that assigns a real number to each outcome in the sample space (denoted by capital letters, e.g., X, Y)

    • Discrete RV: finite or countably infinite values (e.g., number of heads in coin tosses)
    • Continuous RV: uncountably infinite values (e.g., height, weight), described using PDF: P(a≤Z≤b)=∫abfZ(z)dzP(a \leq Z \leq b) = \int_a^b f_Z(z) dz, where fZ(z)f_Z(z) is the probability density function
  • Mean (average): denoted by μ\mu or E(X)E(X), tells where the distribution of XX tends to balance out

    • for discrete RV: E(X)=∑xiP(X=xi)E(X) = \sum x_i P(X=x_i)
    • for continuous RV: E(X)=∫−∞∞xfX(x)dxE(X) = \int_{-\infty}^{\infty} x f_X(x) dx
  • Variance: denoted by σ2\sigma^2 or Var(X)Var(X), measures spread of distribution around the mean

    • Var(X)=E[(X−μ)2]Var(X) = E[(X - \mu)^2] or Var(X)=E(X2)−(E(X))2Var(X) = E(X^2) - (E(X))^2
    • discrete RV: Var(X)=∑(xi−μ)2P(X=xi)Var(X) = \sum (x_i - \mu)^2 P(X=x_i)
    • continuous RV: Var(X)=∫−∞∞(x−μ)2fX(x)dxVar(X) = \int_{-\infty}^{\infty} (x - \mu)^2 f_X(x) dx
    • high variance = unstable prediction
  • Standard Deviation: σ=Var(X)\sigma = \sqrt{Var(X)}

    • Law of Large Numbers (LLN): states that more trials = less randomness = truer picture of reality
  • Probability Distributions: how probabilities are distributed across the possible values of a random variable

    • Discrete Distributions: e.g., Bernoulli, Binomial, Poisson
    • Continuous Distributions: e.g., Uniform, Normal
  • Bernoulli Distribution: denoted X∼Bernoulli(p)X \sim \text{Bernoulli}(p), has 2 outcomes XX random variable, X={1if success0if failureX = \begin{cases} 1 & \text{if success} \\ 0 & \text{if failure} \end{cases}

    • PMF: P(X=k∣p)=pk(1−p)1−kP(X = k | p) = p^k (1-p)^{1-k} for k=0,1k = 0, 1
    • Mean: E(X)=pE(X) = p
    • Variance: Var(X)=p(1−p)Var(X) = p(1-p)
  • Binomial Distribution: denoted X∼Binomial(n,p)X \sim \text{Binomial}(n, p), represents the number of successes in nn repeated Bernoulli trials with probability of success pp

    • PMF: P(X=k∣n,p)=(nk)pk(1−p)n−kP(X = k | n, p) = \binom{n}{k} p^k (1-p)^{n-k} for k=0,1,2,...,nk = 0, 1, 2, ..., n
    • Mean: E(X)=npE(X) = np
    • Variance: Var(X)=np(1−p)Var(X) = np(1-p)
  • Poisson Distribution: denoted X∼Poisson(λ)X \sim \text{Poisson}(\lambda), models the number of events occurring in a fixed interval of time/space with known average rate λ\lambda

    • PMF: P(X=k∣λ)=e−λλkk!P(X = k | \lambda) = \frac{e^{-\lambda} \lambda^k}{k!} for k=0,1,2,...k = 0, 1, 2, ...
    • Mean: E(X)=λE(X) = \lambda
    • Variance: Var(X)=λVar(X) = \lambda
  • Normal (Gaussian) Distribution: denoted X∼N(μ,σ2)X \sim \mathcal{N}(\mu, \sigma^2), models continuous data with a symmetric bell-shaped curve

    • PDF: fX(x∣μ,σ2)=1σ2πe−(x−μ)22σ2f_X(x | \mu, \sigma^2) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x - \mu)^2}{2\sigma^2}} for −∞<x<∞-\infty < x < \infty
    • Mean: E(X)=μE(X) = \mu
    • Variance: Var(X)=σ2Var(X) = \sigma^2
  • Standard Normal Distribution: special case with μ=0\mu = 0 and σ2=1\sigma^2 = 1, denoted Z∼N(0,1)Z \sim \mathcal{N}(0, 1)

    • PDF: fZ(z)=12πe−z22f_Z(z) = \frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}} for −∞<z<∞-\infty < z < \infty
    • Any normal random variable can be standardized to a standard normal variable using Z=X−μσZ = \frac{X - \mu}{\sigma} so P(X≤x)=P(Z≤x−μσ)=Φ(x−μσ)P(X \leq x) = P(Z \leq \frac{x - \mu}{\sigma}) = \Phi(\frac{x - \mu}{\sigma})
  • In scipy.stats, you can use:

    • scipy.stats.binom for Binomial distribution
    • scipy.stats.poisson for Poisson distribution
    • scipy.stats.norm for Normal distribution
    • scipy.stats.bernoulli for Bernoulli distribution
    • Methods:
      • .pmf(k) for PMF of discrete distributions
      • .pdf(x) for PDF of continuous distributions
      • .cdf(x) for CDF of both discrete and continuous distributions
      • .rvs(size=n) to generate n random samples from the distribution