Probability Distribution Functions (PMF, PDF, CDF)

Key Concepts

• Probability Mass Function (PMF)
• Probability Density Function (PDF)
• Cumulative Distribution Function (CDF)
• Discrete Variables
• Continuous Variables
• Gradient
• Integral
• Differential

Probability Distribution Functions: A Detailed Explanation

1. Introduction and Terminology

The video explains probability distribution functions (PDFs), focusing on intuitive understanding rather than complex formulas. It distinguishes between discrete and continuous variables and their associated functions:

• Discrete Variables: Use a Probability Mass Function (PMF), representing the probability of each discrete outcome.
• Continuous Variables: Use a Probability Density Function (PDF), representing the probability density of outcomes on a continuous distribution.
• Both discrete and continuous variables can have a Cumulative Distribution Function (CDF), representing the cumulative probability.

The video clarifies the potential confusion around the term "PDF," noting that it's often used to refer specifically to the probability density function for continuous variables, while "probability distribution function" is a broader term encompassing both PMFs and PDFs.

2. Discrete Variables: The Dice Example

A standard six-sided dice is used to illustrate discrete variables and PMFs.

• Each outcome (1, 2, 3, 4, 5, 6) has a probability of 1/6 (approximately 0.167).
• This equal probability for each outcome is a uniform distribution.

The video then introduces the concept of the Cumulative Distribution Function (CDF) for the dice example.

• The CDF represents the probability of rolling a value less than or equal to a given number.
• For example, the CDF at 4 represents the probability of rolling a 1, 2, 3, or 4.
• The CDF is a cumulative sum of the probabilities from the PMF.
• The final value of the CDF must be 1, representing 100% probability of rolling a 6 or less.

The example is then modified to simulate a rigged dice where the probability of rolling a 3 or 4 is zero.

• This changes the PMF, with the probabilities of 1, 2, 5, and 6 each becoming 0.25.
• The CDF reflects this change with a flat gradient between 2 and 4, indicating no probability mass in that range.
• This demonstrates that a flat section on the CDF indicates a lack of probability mass in the corresponding region of the PMF.

3. Continuous Variables: The Height of Women Example

The height of women is used as an example of a continuous variable.

• It's assumed that female heights are normally distributed with a mean of 165 cm and some standard deviation.
• Height is continuous because it can take on any value within a range (e.g., 165.387 cm).
• The probability distribution is represented by a Probability Density Function (PDF), which has a bell curve shape.

The video addresses a common misconception about PDFs:

• The value of the PDF at a specific point (e.g., 0.04 at 165 cm) does not represent the probability of being exactly that height. It represents the probability density at that point.

The Cumulative Distribution Function (CDF) for height is also introduced.

• The CDF is an S-shaped curve, typical for normal distributions.
• The CDF at a given height represents the proportion of women with heights less than or equal to that height.

4. Linking PDF and CDF for Continuous Variables

The video explains how the PDF and CDF are related.

• At the mean height of 165 cm, the CDF value is 0.5, indicating that 50% of women are shorter than 165 cm.
• At a height of 158 cm, the CDF value is 0.25, indicating that 25% of women are shorter than 158 cm.
• The CDF shows how much of the distribution has been "accumulated" up to a given point.

The video then explains how to derive the PDF from the CDF by looking at the gradient of the CDF.

• The gradient of the CDF at a given point represents the probability density at that point.
• A steeper gradient indicates a higher probability density.
• A flat gradient indicates a low probability density.
• The gradient is approximated by calculating the rise over run using two points close to the point of interest (e.g., 164 cm and 166 cm around 165 cm).
• The video uses Excel to calculate the CDF values at 164 cm and 166 cm, finding a rise of approximately 0.08 over a run of 2, resulting in a gradient of 0.04, which corresponds to the PDF value at 165 cm.

5. Calculus Perspective

For viewers familiar with calculus, the video provides a mathematical representation of the relationship between PDF and CDF.

• The PDF is represented as lowercase f(x).
• The CDF is represented as uppercase F(x).
• The differential of the CDF is equal to the PDF: d/dx F(x) = f(x).
• The integral of the PDF from negative infinity to x is equal to the CDF: ∫(-∞ to x) f(t) dt = F(x).

6. Conclusion

The video concludes by summarizing the relationship between PDF and CDF:

• The gradient of the CDF is the PDF.
• The area under the PDF to the left of a given point is the CDF at that point.
• The video emphasizes the graphical representations of these functions.

The presenter encourages viewers to explore additional resources for differentiation and integration examples and promotes his website, Zed Statistics, for further learning.