Statistical Analysis of Experiments with Python

After Completing This Topic

You'll be able to test differences between two groups with a t-test, compare three or more groups with ANOVA, and correctly interpret p-values in Python.

Why Statistical Testing Matters

A drug-treated group has a mean cell viability of 80%, while the control is 85%. Can you conclude "the drug has an effect"?

Different means don't necessarily mean "there is a difference." Repeating an experiment yields different values each time — this is variation. You need to distinguish whether a 5% difference is a drug effect or just experimental error.

The tool for this judgment is a statistical test, and its result is the p-value.

What Is a p-value?

A p-value is "the probability that a difference this large would occur by chance, assuming there is no real difference."

p-value = 0.03 → "If there's no difference, this result happens by chance 3% of the time"
                → Hard to call it chance → Conclude there IS a difference

p-value = 0.42 → "If there's no difference, this result happens by chance 42% of the time"
                → Could easily be chance → Can't conclude there's a difference

By convention, p < 0.05 is called "statistically significant." However, 0.05 is a convention, not an absolute standard.

t-test: Comparing Two Groups

Independent t-test — compares the means of two independent groups.

import numpy as np
from scipy import stats

# Drug-treated group cell viability (%)
drug = np.array([78, 82, 75, 80, 77, 83, 79, 76])

# Control group cell viability (%)
control = np.array([85, 88, 82, 86, 90, 84, 87, 89])

t_stat, p_value = stats.ttest_ind(drug, control)

print(f"Drug mean: {drug.mean():.1f}%")
print(f"Control mean: {control.mean():.1f}%")
print(f"t-statistic: {t_stat:.3f}")
print(f"p-value: {p_value:.4f}")

if p_value < 0.05:
    print("→ Statistically significant difference (p < 0.05)")
else:
    print("→ No significant difference")

assert p_value < 0.05

stats.ttest_ind() — independent t-test. Used when the two groups are from different samples.

Paired t-test: Before/After on the Same Samples

When comparing pre/post treatment in the same patients, use a paired t-test:

from scipy import stats
import numpy as np

before = np.array([120, 135, 128, 142, 138, 125])
after = np.array([115, 128, 122, 130, 132, 118])

t_stat, p_value = stats.ttest_rel(before, after)

print(f"Pre-treatment mean: {before.mean():.1f}")
print(f"Post-treatment mean: {after.mean():.1f}")
print(f"p-value: {p_value:.4f}")

ttest_rel() — related (paired) t-test. Compares measurements from the same subject.

ANOVA: Comparing Three or More Groups

What if you want to compare cell proliferation across three media (DMEM, RPMI, MEM)? Since t-tests only compare two groups, you use ANOVA (Analysis of Variance).

from scipy import stats
import numpy as np

dmem = np.array([1.2, 1.4, 1.3, 1.5, 1.1])
rpmi = np.array([1.8, 1.7, 1.9, 2.0, 1.6])
mem = np.array([1.0, 1.1, 0.9, 1.2, 1.0])

f_stat, p_value = stats.f_oneway(dmem, rpmi, mem)

print(f"DMEM mean: {dmem.mean():.2f}")
print(f"RPMI mean: {rpmi.mean():.2f}")
print(f"MEM mean: {mem.mean():.2f}")
print(f"F-statistic: {f_stat:.3f}")
print(f"p-value: {p_value:.6f}")

assert p_value < 0.05

A significant ANOVA p-value means "at least one of the three groups is different." To find which group differs, use a post-hoc test.

Visualizing Results: Box Plots

Numbers alone make it hard to see distributions. Visualize with box plots:

import matplotlib.pyplot as plt
import numpy as np

drug = np.array([78, 82, 75, 80, 77, 83, 79, 76])
control = np.array([85, 88, 82, 86, 90, 84, 87, 89])

fig, ax = plt.subplots(figsize=(6, 4))
ax.boxplot([drug, control], labels=["Drug", "Control"])
ax.set_ylabel("Cell Viability (%)")
ax.set_title("Drug vs Control")
plt.tight_layout()
plt.savefig("drug_vs_control.png", dpi=150)
plt.show()

Box plots show the median, interquartile range, and outliers at a glance. They're one of the most common graph types in paper figures.

Caveats: Pitfalls of p-values

Points to note when interpreting p-values:

1. p < 0.05 doesn't mean an "important" difference

Statistical significance and practical significance are different. With very large sample sizes, even tiny differences can yield p < 0.05. Always ask: "Is this difference biologically meaningful?"

2. p > 0.05 doesn't mean "no difference"

It means "we didn't find evidence of a difference." With too few samples, real differences may go undetected.

3. Multiple comparisons problem

Testing 20,000 genes at once means 5% — that's 1,000 genes — will be p < 0.05 by chance. In this case, Bonferroni correction or FDR (False Discovery Rate) correction is needed.

Try It Yourself (Faded Example)

Fill in the blanks to perform a t-test between two groups.

from  import stats
import numpy as np
treated = np.array([4.2, 3.8, 4.5, 4.1, 3.9])
control = np.array([5.1, 5.3, 4.9, 5.0, 5.2])
t_stat, p_value = stats.ind(treated, control)
print(f"p-value: {p_value:.4f}")
if p_value < :
  print("Significant difference")

Common Errors & Solutions

Q: ModuleNotFoundError: No module named 'scipy'

Install with pip install scipy. It's already installed on Google Colab.

Q: Should I use a t-test or ANOVA?

Use t-test for 2 groups, ANOVA for 3 or more. Running t-tests 3 times on 3 groups (A-B, A-C, B-C) creates a multiple comparisons problem, so use ANOVA instead.

Q: What if my data isn't normally distributed?

t-tests and ANOVA assume data roughly follows a normal distribution. For non-normal data, use non-parametric tests: Mann-Whitney U test (stats.mannwhitneyu()), Kruskal-Wallis test (stats.kruskal()).

Q: nan appears in results

Your data may contain missing values (NaN). Use nan-ignoring functions like np.nanmean(data), or remove missing values with df.dropna() in Pandas before testing.