Comparing mean yield for media A, B, C, and D with every pairwise t-test creates six tests. At α=.05 for each, the opportunity for at least one false positive somewhere grows.

ANOVA calculates a ratio of variation rather than comparing means directly

The one-way ANOVA null is H0: μ₁=μ₂=…=μk. Its alternative is not “every mean differs,” but at least one mean differs. This overall question is the omnibus test.

Partition the total variation of observations around the grand mean into:

• between-group variation: how far group means are from the grand mean
• within-group variation: how observations vary around their own group mean

SS_total = SS_between + SS_within

Sum of squares (SS) naturally grows with the number of observations. Divide each SS by its degrees of freedom (df) to form mean squares (MS), then calculate F=MS_between/MS_within. Under H0, both MS target the same error variance, so F tends to lie near 1. F grows when separation among group means exceeds within-group noise.

Each ANOVA-table column connects in one statement

With k groups and N total independent observations:

• between df = k−1
• within df = N−k
• MS_between=SS_between/(k−1)
• MS_within=SS_within/(N−k)
• F=MS_between/MS_within

The ANOVA p-value is the probability, under H0 and the model, of an F at least this large. A significant F is evidence that a difference exists somewhere; it does not identify which group, how large the difference is, or whether all groups differ.

Repeated t-tests versus post hoc comparisons

If a specific comparison was the scientific question before analysis, state it as a planned comparison. If all pairs are explored after viewing the data, a family-wise-error-controlled procedure such as Tukey HSD is an appropriate candidate. Use methods matched to the purpose, such as Dunnett for comparing several treatments with one control.

Adjustment can widen each CI. This is not a penalty: it is the cost of making one error promise across many opportunities. Selecting arbitrary pairwise p-values when ANOVA is non-significant needs justification from the pre-specified plan and method.

Assumptions are not a collection of per-group normality p-values

Classical one-way ANOVA assumes independent errors, roughly normal errors within groups, and equal variances. Balanced designs have some robustness, but problems grow with small n, strong skew, outliers, or an inverse relationship between variance and n.

Welch ANOVA is an alternative less dependent on equal variances. Rank-based Kruskal–Wallis is not an assumption-free version of mean ANOVA; its distributional and location interpretations differ. Repeated-measures or clustered data require a structure-matched analysis such as a mixed model, not direct insertion into a one-way ANOVA table.

In-Silico Lab: verify SS by hand

The synthetic means of three groups are 10, 10+d, and 10+2d.

1. At d=0, see that F is not fixed at exactly 1.
2. Increase d and inspect the tendency for SS_between and F to grow.
3. Change the seed and explain why SS_within and F change.
4. Use the calculation engine to verify that displayed SS_between + SS_within equals total SS.

Start JMP ANOVA interpretation from the decomposition table

Read Analysis of Variance from left to right: Source, DF, Sum of Squares, Mean Square, F Ratio, and Prob > F. Do not declare a model good from Summary of Fit R² alone. Connect it to raw group data and residuals, Welch results, and simultaneous CIs for post hoc comparisons.

Example result statement

Mean yield for four independent batch conditions was compared by one-way ANOVA (n=8 each). The condition effect was F(3,28)=5.42, p=.004. In prespecified Dunnett comparisons versus control, B−A was 3.1 percentage points (simultaneous 95% CI 0.8–5.4), while the C and D CIs contained zero. Residual plots showed no marked curvature or increasing-variance pattern.

Takeaways

• The ANOVA alternative means at least one mean differs.
• Total SS partitions exactly into between- and within-group SS.
• F is the ratio of two mean squares.
• An omnibus p-value does not say which groups differ.
• Planned and adjusted post hoc comparisons differ in timing and scope of the question.
• Do not reduce repeated or clustered structures to independent one-way ANOVA.

The next unit addresses the error of calling a non-significant result “the same,” then introduces equivalence margins and TOST.

Official supplementary resources

• NIST/SEMATECH · One-Factor ANOVA
• NIST/SEMATECH · Check of assumptions

The values, figures, and Lab in this article are educational synthetic material.