Log In to continue learning
Now let's explore variance. Variance is a more complex calculation than range and IQR, and is the foundation of many other statistics, including the standard deviation. Remember that the whole point of calculating variability is to quantify how spread out the values are. You can step through the calculation for variance using the movie Ratings from 1934. The idea behind variance is to find the average squared distance of the individual values from the mean. The process for calculating it involves looking at each value in your sample, in this case each movie rating, finding the difference between that value and the mean, squaring that difference, and then repeat that for all of the values. Then you sum all of these squared differences and divide by one less than your sample size. We'll discuss the one less point in more detail shortly. The squared differences are also sometimes called square deviations, with deviation just meaning how different each value is from the mean. In the case of the 1934 movies, you can start with Cleopatra, the lowest rated movie. Start with the movie's rating, 6.9. Subtract the mean of 7.5 from it, giving you negative 0.6. Then square that, 0.36. Great. Pause there. Now you need to do the same for each other movie. So for example, The Merry Widow, with a rating of 7.5. Subtract the mean from that and you get zero. Squared is still zero. It happened one night, which received a rating of 8.3. Subtract the mean from 8.3 and you get 0.8. Squared is 0.64. And you'll do this for each of the values. Here's how it looks. Notice how each of these values is either zero or positive. That's one purpose of the squaring operation, to make sure that when you add these values all up, you don't have positive and negative values canceling each other out. Another effect of squaring is that it emphasizes larger deviations. So observations that are far away from the mean contribute much more significantly to the overall variance. Your final step is to divide the total of these squared differences by the sample size n minus 1, which in this case is 23. The result you get is 0.13. A quick note here. You might be wondering why you don't divide by just n and calculate the average sum of squared deviations. That's great intuition. However, n minus 1 is a minor adjustment that helps make your calculation more accurate when working with the sample rather than the entire population. By using n minus 1, you're applying a small correction that makes your estimate more reliable, especially for smaller sample sizes. Notice that n minus 1 is a little smaller, which makes your estimate for the variance a little higher. This helps your estimate account for a little bit more uncertainty in this statistic. Compare the variance you just calculated with the variance for 1966, which is 0.42, almost three times higher. So this calculation is confirming your intuition that the movie ratings in 1966 were much more spread out. Now that you've calculated it through, let's formalize the definition for sample variance. Each value in your sample is XFI, and you're subtracting the sample mean, X-bar, from that. Square this up. Now do you remember the notation for summing up all these individual values? That would be sigma, the big E-looking symbol. And finally, divide by n minus 1. Now one issue with interpreting variance is that because you squared the differences between the mean and each value, the variance is not in the same units as your data. Right now, your variance is in the units, points squared. Awkward. However, if you just take the square root, you'll get back to having a calculation in points, in this case, 0.355. This square root of the variance is called the standard deviation. The sample standard deviation is written as S. So you already saw X-bar for the sample mean. Now you have S for the sample standard deviation. Sample variance is written S-squared, since it's the standard deviation squared. Let's compare with the standard deviation for movie ratings from 1966, which is 0.655. We have the same interpretation as before. The 1966 has a higher standard deviation. Now you can visualize these statistics on the histograms from before. The dashed red lines each indicate the mean minus one standard deviation on the left and the mean plus one standard deviation on the right. The difference on the left is much more narrow, while the difference on the right is wider. It looks like the values in the 1966 histogram are almost pulling those dashed lines outward, which mathematically is exactly what happens. In practice, you'll see both standard deviation and variance, since variance is used to derive multiple other calculations, and standard deviation is commonly used to characterize your distributions. Great work on calculating these measures of variability. Join me in the next video to better understand the third key measure of your distribution, skewness. I'll see you there.