The Law of Large Numbers
The Law of Large Numbers
In this tutorial, you will learn about the law of large numbers in probability theory.
Have you ever flipped a coin a few times and noticed that sometimes you get more heads than tails, or vice versa? But if you keep flipping the coin again and again—say hundreds or thousands of times—you’ll find that the number of heads and tails tends to even out.
This amazing and intuitive idea is captured in a fundamental concept in probability theory known as The Law of Large Numbers.
What is The Law of Large Numbers?
The Law of Large Numbers (LLN) is a statistical principle that states: as the number of trials or experiments increases, the average of the results becomes closer to the expected value (or theoretical probability). It bridges the gap between probability theory and real-world observation.
This law helps us understand why long-term trends in chance processes become predictable, even if individual outcomes are random and unpredictable.
Mathematical Representation
Let’s say we have a sequence of random variables: X1, X2, X3, …, Xn that are independent and identically distributed (i.i.d), each with an expected value E[X] = μ.
Then, the sample average is given by:
Sn = (X1 + X2 + ... + Xn) / n
According to the Law of Large Numbers:
As n → ∞, Sn → μ
This means the sample average Sn converges to the expected value μ as the number of trials n becomes very large.
Simple Example
Imagine you’re rolling a fair six-sided die. The possible outcomes are 1, 2, 3, 4, 5, and 6. The average of these outcomes (the expected value) is 3.5. But if you roll the die just a few times, your average might be much higher or lower than 3.5. Maybe you get 6, 6, 1—then your average is 4.33, which is off from 3.5.
However, if you roll the die 1000 times, the average of those 1000 rolls will get very close to 3.5. That’s the Law of Large Numbers in action—it shows that randomness averages out when you collect enough data.
Real-Life Examples
Some of the real-life examples are as follows:
- Insurance: Insurance companies use LLN to predict the average number of claims and set premiums accordingly. Though individual events are uncertain, the average over thousands of customers is predictable.
- Casino Games: Casinos rely on LLN to ensure that, over many games, they make a profit close to the expected edge they have in each game.
- Polling: Opinion polls take a sample of the population. As the sample size increases, the average opinion tends to reflect the overall population more accurately.
- Quality Control: In manufacturing, checking a large batch of items gives a reliable average defect rate, even if individual items vary.
Random Experiment Flowchart
A simulation experiment is shown here:
