Confidence Intervals
An interval of 4 plus or minus 2
A Confidence Interval is a range of values we are fairly sure our true value lies in.
Example: Average Height
We measure the heights of 40 randomly chosen men, and get a mean height of 175cm,
We also know the standard deviation of men's heights is 20cm.
The 95% Confidence Interval (we show how to calculate it later) is:
The "±" means "plus or minus", so 175cm ± 6.2cm means
- 175cm − 6.2cm = 168.8cm to
- 175cm + 6.2cm = 181.2cm
And our result says the true mean of ALL men (if we could measure all their heights) is likely to be between 168.8cm and 181.2cm
But it might not be!
The "95%" says 95% of experiments like we just did will include the true mean, but 5% won't.
Imagine we repeat this sampling process again and again, then about 19 out of 20 would contain the true mean, and about 1 out of 20 would miss it. This might be the one that misses it!
Calculating the Confidence Interval
Step 1: collect the sample information
- the number of observations n
- the mean X
- and the sample standard deviation s
Note: Ideally we would use the population standard deviation σ, but we usually don't know it.
So instead for larger samples (about n ≥ 30) using the sample standard deviation s works well enough.
Using our example:
- number of observations n = 40
- mean X = 175
- standard deviation s = 20
Step 2: choose the confidence level
Common choices are 95% or 99%. A 95% confidence interval means that this method would capture the true mean about 95% of the time if we repeated the sampling many times.
Find the corresponding Z-value here:
| Confidence Interval |
Z |
| 80% | 1.282 |
| 85% | 1.440 |
| 90% | 1.645 |
| 95% | 1.960 |
| 99% | 2.576 |
| 99.5% | 2.807 |
| 99.9% | 3.291 |
For a 95% confidence interval, the Z-value is 1.960
Step 3: use that Z value in this formula for the Confidence Interval:
X ± Zs√n
- X is the mean
- Z is the chosen Z-value from the table above
- s is the standard deviation
- n is the number of observations
And we have:
175 ± 1.960 × 20√40
Which is:
175cm ± 6.20cm
In other words: from 168.8cm to 181.2cm
The value after the ± is called the margin of error
The margin of error in our example is 20√40 = 6.20cm
Calculator
We have a Confidence Interval Calculator to make life easier for you.
Simulator
We also have a very interesting Normal Distribution Simulator where we can start with some theoretical "true" mean and standard deviation, and then take random samples.
It helps us to understand how random samples can sometimes be very good or bad at representing the underlying true values.
Another Example
Example: Apple Orchard
Are the apples big enough?
There are hundreds of apples on the trees, so you randomly choose just 46 apples and get:
- a Mean of 86 g
- a Standard Deviation of 6.2 g
So let's calculate:
X ± Zs√n
We know:
- X is the mean = 86
- Z is the Z-value = 1.960 (from the table above for 95%)
- s is the standard deviation = 6.2
- n is the number of observations = 46
86 ± 1.960 × 6.2√46 = 86 ± 1.79
So the true mean (of all the hundreds of apples) is likely to be between 84.21 and 87.79
True Mean
Now imagine we get to pick ALL the apples straight away, and get them ALL measured by the packing machine (this is a luxury not normally found in statistics!)
And the true mean turns out to be 84.9
Let's lay all the apples on the ground from smallest to largest:
Each apple is a green dot,
our observations are marked blue
Our result was not exact ... it is random after all ... but the true mean is inside our confidence interval of 86 ± 1.79 (in other words 84.21 to 87.79)
Now the true mean might not be inside the confidence interval, but in 95% of the cases it will be!
95% of all "95% Confidence Intervals" will include the true mean.
Maybe we had this sample, with a mean of 83.5:
Each apple is a green dot,
our observations are marked purple
That does not include the true mean. That can happen about 5% of the time for a 95% confidence interval.
So how do we know if our sample is one of the "lucky" 95% or the unlucky 5%? Unless we get to measure the whole population like above we simply don't know.
This is the risk in sampling, we might have a "bad" sample.
Example in Research
Here is Confidence Interval used in actual research on extra exercise for older people:
What is it saying? Looking at the "Male" row we see:
- 1,226 men (47.6% of all participants)
- had a "HR" (see below) with a mean of 0.92,
- and a 95% Confidence Interval (95% CI) of 0.88 to 0.97 (which is also 0.92 ± 0.05)
"HR" stands for Hazard Ratio: a common measure in medical research. An HR below 1 means lower risk (a better outcome), while above 1 means greater risk (a worse outcome).
So for the men’s HR: we are 95% confident that the true population HR lies between 0.88 and 0.97. The entire interval is below 1, so the reduction in risk is statistically significant (p < 0.05). In plain language: the extra exercise appears to give a real benefit for men.
But this is not the case for women in the same study. Their 95% CI was 0.90–1.03 (which includes 1), so no statistically significant benefit was found.
Standard Normal Distribution
It is all based on the idea of the Standard Normal Distribution, where the Z value is the "Z-score"
For example the Z for 95% is 1.960, and here we see the range from -1.96 to +1.96 includes 95% of all values:
From -1.96 to +1.96 standard deviations is 95%
Applying that to our sample looks like this:
Also from -1.96 to +1.96 standard deviations, so includes 95%
Conclusion
The Confidence Interval is based on Mean and Standard Deviation. Its formula is:
X ± Zs√n
Where:
- X is the mean
- Z is the Z-value from the table below
- s is the standard deviation
- n is the number of observations