Lesson 8 — Post Hoc Tests
When ANOVA finds a significant F, we know that not all group means are equal.
But ANOVA does not tell us which groups differ.
For that, we need post hoc tests (Latin: after this).
They compare pairs of group means while controlling for the increased chance of error.
Tukey’s Honestly Significant Difference (HSD)
When to Use:
- Equal group sizes
- Pairwise comparisons after one-way ANOVA
Formula:
$$\text{HSD} = q \sqrt{\frac{MS_{\text{within}}}{n}}$$
In words:
$$\text{HSD} = \text{Studentized range statistic } q \times \sqrt{\frac{\text{mean square within groups}}{\text{sample size per group}}}$$
If the difference between two means ≥ HSD, they are significantly different.
Example:
3 groups, n = 10 each, $$MS_{\text{within}} = 16.7$$, critical $$q = 3.5$$.
$$\text{HSD} = 3.5 \times \sqrt{\tfrac{16.7}{10}} = 3.5 \times 1.29 = 4.52$$
So any pair of means that differ by 4.52 or more is significant.
Bonferroni Correction
When to Use:
- Simple and conservative
- Divide significance level by number of comparisons
Formula:
$$\alpha' = \frac{\alpha}{m}$$
In words:
$$\text{adjusted significance level} = \frac{\text{original significance level}}{\text{number of comparisons}}$$
Example: If α = 0.05 and 10 comparisons, α′ = 0.005 per test.
Scheffé Test
When to Use:
- Unequal sample sizes
- Most conservative post hoc test
Formula (summary):
Scheffé’s critical F = (k – 1) × F(critical, df_between, df_within).
Definition
- Post hoc test: statistical test used after ANOVA to identify which means differ
- Tukey HSD: balanced groups, pairwise
- Bonferroni: adjusts α for multiple comparisons
- Scheffé: conservative, flexible for unequal n
Visual Placeholders
Figure 8.1 — Tukey HSD example: three group means with horizontal bars showing which pairs differ.
Figure 8.2 — Bonferroni correction illustration: α = 0.05 split into smaller pieces.
Why This Matters
ANOVA tells us there is a difference somewhere.
Post hoc tests tell us where.
They protect against false positives while allowing multiple group comparisons.
Practice self-test quiz
In the space below, please find practice problems and self-test quizzes. For full access, please signup free.