A difference test between the means of A and B gives p=.24. May we write, “There is no significant difference, so the two conditions are the same”? No. With a small sample and large variation, even a large difference can remain undetected.

Difference tests and equivalence tests have opposite null hypotheses

A difference test begins with H0: difference=0 and seeks evidence that a difference exists. Failure to reject means the data are compatible with 0; it does not prove that the difference lies within a small range.

An equivalence test first specifies a practically negligible boundary, −Δ and +Δ. If the mean difference is δ=μB−μA:

• H0: δ≤−Δ or δ≥+Δ — the difference lies outside the acceptable range.
• H1: −Δ<δ<+Δ — the difference lies inside the acceptable range.

TOST tests the following two null hypotheses separately at one-sided α.

1. Reject H0 that B is at least Δ lower than A.
2. Reject H0 that B is at least Δ higher than A.

Both must be rejected before evidence for equivalence exists. If either fails, the result may establish non-equivalence, or it may simply be inconclusive.

A TOST at α=.05 corresponds to a 90% CI

In general, if each one-sided test uses α, the same decision can be made by checking whether the (1−2α)×100% two-sided CI for the difference lies entirely inside the equivalence margins. At α=.05, this is a 90% CI.

• the full CI lies within −Δ and +Δ: evidence for equivalence
• the CI crosses a margin but includes 0: inconclusive for both difference and equivalence
• the CI is far from 0 and outside a margin: a meaningful difference is possible

Do not use a conventional 95% CI interchangeably with TOST p-values. A 95% CI may also be presented for estimation, but state clearly which CI corresponds to which decision.

See the CI and margins before two small p-values

TOST output contains a lower-test and an upper-test p-value. The larger p-value must be below α for both conditions to pass. But placing the CI and margins on the same axis shows more directly which boundary failed and in which direction the effect lies.

Equivalence does not automatically establish sameness of an entire process, specification conformity of an individual product, equality of variances, or interchangeability. State equivalence narrowly: for which endpoint, population, time window, and effect scale?

In-Silico Lab: the same true difference can lead to different conclusions at different n

1. Compare n=8 and n=100 with a true difference of 0 and Δ=1.
2. Observe the region where a small n supports neither a difference test nor an equivalence test.
3. Raise the true difference to 1.5 and see whether the CI center moves outside a margin.
4. Explain why widening Δ without changing the data adds no scientific evidence.

The Lab is an educational one-sample difference model with known σ. Actual two-sample, paired, proportion, or log-transformed endpoints need design-appropriate SEs and CIs.

JMP displays margins and CIs but does not justify the margin

Historical feed, biosimilar examples, and regulatory limits from earlier lectures are not reused as public answers. This article uses a synthetic process difference; JMP is limited to displaying the relationship among Practical Difference, difference estimates, the two one-sided results, and the CI.

Example result statement

The prespecified equivalence margin was ±2.0 percentage points for mean activity difference. The independent-batch difference B−A was 0.4, and the 90% CI corresponding to an α=.05 TOST was −1.1 to 1.9, entirely within the margins. Both one-sided tests were rejected, providing evidence for equivalence for this endpoint and analysis population. This result does not establish individual-batch specification conformity or equivalence of other quality attributes.

When evidence for equivalence is insufficient, do not automatically replace it with “non-equivalent.” Distinguish an inconclusive result from a clear margin exceedance according to the CI position.

At the end of this unit

• A nonsignificant difference test does not prove equivalence.
• The equivalence null hypothesis says the difference is outside the margins.
• TOST requires rejection of both one-sided null hypotheses.
• An α=.05 TOST corresponds to a 90% CI lying entirely inside the margins.
• Δ is justified before data using domain evidence.
• State conclusions with a limited endpoint and scope.

The next unit links effect, variation, α, power, and sample size so that a study has enough precision to reach an active conclusion such as equivalence.

Official supplementary resources

• FDA · Statistical Approaches to Establishing Bioequivalence
• JMP · Equivalence Tests

This article explains equivalence principles and does not approve regulatory or clinical study designs.