Student's T-Test

(Named after William Sealy Gosset, who published under the name "Student" to respect his employer's privacy.)

Student's t-test helps us compare two sets of data to see if the difference between their means is just random chance.

There are two main types:

• Independent Samples T-Test: to compare two distinct groups where members of one are not related to the other.
• Paired Samples T-Test: when the same participants are in both groups being compared, such as before-and-after observations.

The data should be normally distributed, and is not reliable if the sample sizes are too small or the variances are very different.

Note: you migh also like to investigate the Chi-Square Test.

Independent Samples T-Test

We typically use this formula (but there are other formulas):

Where:

• x₁ and x₂ are sample means
• s₁² and s₂² are sample variances
• n₁ and n₂ are sample sizes

Note: we use s² for variance, because the variance is the square of the standard deviation s.

To do the t-test:

• Calculate mean, variance and samples size for each set of sample data
• Use the formula above to calculate "t"
• Then we look up a special table to find out how significant that "t" value is

In effect the t-value is a measure of how much the means of the two groups differ from each other compared to the variability of the data.

A higher t-value suggests a more significant difference between the groups (ie less likely due to random chance).

Let's try an example:

Example: Battery Life

This is an experiment you can do yourself (at the cost of 20 batteries).

Two batches of identical batteries are tested:

• One group of 10 were tested in a cool environment
• Another group of 10 were tested in a hot environment

The batteries power identical devices until they run out of charge, with these results (in hours):

• Cool Group: 85, 90, 76, 88, 93, 87, 91, 89, 95, 85
• Hot Group: 78, 82, 79, 88, 91, 85, 90, 87, 84, 86

Let's calculate the mean for each:

• Cool Group Mean = 85+90+76+88+93+87+91+89+95+8510 = 87.9 hours
• Hot Group Mean = 78+82+79+88+91+85+90+87+84+8610 = 85 hours

The Cool Group lasted, on average, about 3 hours longer.

But is this a significant result, or just random in nature?

Now, it's time to calculate the t-statistic.

Our next step is to calculate Variances.

Variance is a measure of how much the data points spread out from the mean. We calculate it by taking the average of the squared differences between each data point, as shown here:

For the Cool Group:

• n = 10
• Mean = 87.9
• Variance = (85-87.9)²+(90-87.9)²+(76-87.9)²+(88-87.9)²+(93-87.9)²+(87-87.9)²+(91-87.9)²+(89-87.9)²+(95-87.9)²+(85-87.9)²9 = 27.88

For the Hot Group:

• n = 10
• Mean = 85
• Variance = (78-85)²+(82-85)²+(79-85)²+(88-85)²+(91-85)²+(85-85)²+(90-85)²+(87-85)²+(84-85)²+(86-85)²9 = 18.89

Start with:

t = x₁ − x₂√(s₁²/n₁ + s₂²/n₂)

Put in our values:

87.9 − 85√(27.88/10 + 18.89/10)

Calculate:

t = 1.341

Now let's compare our t-value to the "critical t-value" from the Student's t-test Table at a chosen significance level, typically 0.05 for a 95% confidence level with degrees of freedom df = n₁ + n₂ − 2 = 18, and using the two-tails value, as the change in battery life could theoretically go either way.

The table shows us that for 0.05 and df=18 we get a "critical t-value" of 2.101

Our actual t value is 1.341, and being less than 2.101 our result does not pass the 95% confidence level.

So there is not enough evidence to say there is a significant difference between the cool and hot groups.

Hypothesis

The t-test is often done more formally using the idea of a hypothesis and null hypothesis.

But we usually take the opposing point of view:

The null hypothesis is the baseline assumption for statistical analysis

Rejecting the null hypothesis gives credibility to (but does not prove) our original hypothesis. In other words if we can show the skeptic is likely wrong, we can suggest our original idea might be right.

Hypothesis Steps

1. Formulate the null hypothesis (H₀): There is no significant difference between the two groups.
2. Formulate the alternative hypothesis (H_a): There is a significant difference between the two groups.
3. Choose a significance level (commonly 0.05), which is the probability of rejecting the null hypothesis when it is actually true.
4. Collect the data and calculate the t-statistic based on the two sample means, standard deviations, and sample sizes.
5. Compare the calculated t-statistic to the critical value from the t-distribution table to determine whether to accept or reject the null hypothesis.

Let's do our original example that way!

Example: Battery Life

Null Hypothesis (H₀): The difference between a cool and hot environment has no effect on a battery's running time

Alternative Hypothesis (H_a): The difference between a cool and hot environment will affect a battery's running time

Using the values calculated earlier:

From before:

87.9 − 85√(27.88²/10 + 18.89²/10)

Calculate:

t = 1.341

From the Student's t-test Table for 0.05 (which is a 95% confidence level) and df=18 we get a "critical t-value" of 2.101

Since our calculated t-value of 1.341 is less than the critical t-value of 2.101, we do not have enough evidence to reject the null hypothesis.

We cannot reject the null hypothesis

This means that based on our sample, we cannot confidently say a statistically significant difference exists.

In everyday terms, it's as if we're saying, "Based on what we've seen, we can't conclude that the change is anything more than just random chance."

Paired Samples T-Test

A Paired samples t-test, also known as a dependent samples t-test, is a statistical test that is used to compare two means (averages) when the data is paired in some way. Such as a before and after trial.

So the same subjects are involved in both sets of data.

Step-by-Step Calculation

To do the Paired Samples t-test:

• Calculate the difference (d) for each pair of scores
• Calculate the mean difference (d)
• Calculate the variance of the differences (var)
• Calculate the standard deviation of the differences (s)
• Calculate the t-statistic.
• Determine the degrees of freedom (df)
• Compare the calculated t-statistic to the critical value from the t-distribution table to determine whether to accept or reject the null hypothesis.

Now let's compare our t value to the critical t-value in the table.

• The degrees of freedom (df) for a paired t-test is n − 1: df = 7−1 = 6
• For a two-tailed test with df=6 at the 0.05 level, we find the critical t-value is 2.447
• Our value of 2.78 is above that, so the Null Hypothesis fails

It is unlikely that the Null Hypothesis is true (less than 5% chance), so we have good reason to believe that the alternative hypothesis "The special math program does affect test scores" is true

One-Tail and Two-Tail T-Tests

• One-Tail T-Test: Tests for a significant effect in one specific direction (either greater than or less than).
• Two-Tail T-Test: Tests for a significant effect in both directions (either greater or less than), without committing to one specific direction before the test.

Choose a one-tail test if you have a specific hypothesis about the direction of the effect. Use a two-tail test if you're interested in detecting any significant difference, regardless of direction.

Single Set of Data

For one set of data we can calculate a t value like this: