Distribution Visualization and Outliers

After Completing This Topic

You will be able to explain what information histograms, dot plots, and box plots each reveal. Rather than immediately classifying conspicuous observations as errors or candidates for removal, you will be able to connect them to data quality checks and a reexamination of the research question.

Starting Point: What the Table of Means Does Not Show

In stat-004 we calculated center and spread. However, a summary table compresses the ordering of values and how densely they cluster. Even if two conditions have similar means and IQRs, one of them may have multiple peaks or a long tail.

The purpose of visualization is not to produce a "pretty picture." It is to visually inspect the distribution of values, group differences, missingness, and abnormal ranges while preserving the meaning of a single row and the unit of measurement.

The questions behind three plots

Dot plot

Displaying each observation as a point lets you check the actual number and location of the data. When the sample is not large, it is a good way to explain whether a single point is one unit of analysis or a technical replicate. If points overlap, you can use jitter to spread them randomly, but you need to explain that the displaced positions do not represent new measured values.

Histogram

It shows the frequency of values by interval. Because the shape can change depending on the bin width, do not interpret a particular shape as a fixed structure of nature. Use it as a starting point for exploring the possibility of long tails, empty intervals, or multiple peaks.

Box plot

Along with the median, quartiles, and range, it summarizes points that fall extremely far away. A point outside the box plot is not a verdict of "an error that must be removed." It is a visual signal that the observation lies far from that particular rule.

Plotting distributions with Python

The code below uses the same self-contained synthetic structure as stat-004. Before moving on to visualization, you should check the data types, missing values, and unit of analysis for condition and outcome.

import matplotlib.pyplot as plt
import seaborn as sns

fig, axes = plt.subplots(1, 2, figsize=(9, 3.5))
sns.stripplot(data=data, x="condition", y="outcome", ax=axes[0])
sns.boxplot(data=data, x="condition", y="outcome", ax=axes[1])
axes[0].set_title("individual observations")
axes[1].set_title("distribution summary")
fig.tight_layout()

This code runs on the premise that data is the DataFrame created in the previous article. The strip plot shows the position of individual values, while the box plot shows the summary of the distribution and the distant points. Axis names and arbitrary units should be changed to match your actual data contract. If you produce an image file, record the generating code, environment, and input hash in the manifest.

The same principle applies when you add a histogram.

fig, ax = plt.subplots(figsize=(5, 3.5))
sns.histplot(
    data=data,
    x="outcome",
    hue="condition",
    bins=8,
    element="step",
    stat="count",
    common_norm=False,
    ax=ax,
)
ax.set_xlabel("outcome (arbitrary unit)")
ax.set_ylabel("number of observations")
fig.tight_layout()

The choice of bins affects the shape you see. So check whether the pattern holds even when you change the bin width, and do not declare the essential nature of a distribution from a single histogram alone. When overlaying groups with hue, provide the legend together with the number of observations so that color alone does not hide the unit of analysis or the sample size.

For visualization results, the conditions under which they were generated matter more than the image. You must record the figure's title, axis units, the subset used, the rule for excluding missing values, and the number of observations per group so that someone else can regenerate the same figure.

After you find an outlier

When you notice a striking value, check the records first rather than deleting the data.

1. Confirm the row ID and the unit of analysis for the value.
2. Check whether that value appears in the entry, transformation, and measurement logs.
3. Inspect whether the units, decimal places, dates, and group labels are correct.
4. If it is a measurement error, preserve the basis for the correction and the original record.
5. If it is genuine variation, record the inclusion criteria and a sensitivity analysis instead of deleting it arbitrarily.

This sequence links the visual judgment that a particular value looks unusual to the research record. A rule like "remove anything outside the box" cannot explain what caused the observation.

Separating judgments through sensitivity analysis

Sometimes checking the measurement log still does not settle how a value should be handled. In that case, you can preserve the raw data and compare the result obtained under a pre-specified rule with the result obtained under a separate rule. This comparison is not a device that automatically picks which result is "real"; it is a way of revealing how much the conclusion depends on particular observations.

However, if you run a sensitivity analysis and then select only the rule whose results look better, the purpose of the analysis has changed. You must record the exclusion criteria, the reason for applying them, and the difference in results together, and explain which criterion fits the research question.

Showing nested structure in the figure

If multiple technical replicates came from the same biological replicate, do not present every point as if it were an equally independent sample. You can use connecting lines, colors, facets, or summary markers per higher-level unit, but visual decoration does not substitute for the structure of independence. Stating in the figure caption "what a single point means" comes first.

Returning to the Research Question for Interpretation

Distribution visualization sharpens the question of what form the data took under each condition. You can check whether a difference in center appears across all ranges, whether a few values are driving the result, and whether measurement units have been mixed.

However, a single figure does not establish the causal effect of a condition or its biological importance. Visualization is a stage for checking quality and hypotheses; it does not replace independence, uncertainty, or study design.

The Order in Which to Read a Figure

Do not make a figure and immediately declare a difference. Read it in the following order.

1. Check the units and transformations of the axes.
2. Check which unit of analysis each point represents.
3. Check the number of observations and missing values per condition.
4. Separately examine center, spread, tails, and the possibility of multiple peaks.
5. Check in the raw data and records how much a particular point influences the result.

This order links visual impressions to the data contract. For example, even if group B's boxplot looks wider, if B has fewer observations or some units are missing, that fact must be recorded before reading it as a comparison under identical conditions. Color contrast between groups aids interpretation, but color alone does not convey the meaning, order, or superiority of conditions.

Accessibility and Reporting

If groups are distinguished by color alone, information can disappear under color vision differences or in black-and-white printing. Use legend text, axis labels, point shapes, or panel separations together with color. In the image description, write what the data mean, what unit a single point represents, and whether unprocessed observations were included.

When placing a plot into a report, do not simply copy the figure file; link it to the generating code and the data version. Record figure_id, the input hash, the code version, and the generation time so that a different subset is not plotted under the same filename. A visualization is also an analysis output, so it is part of the provenance.

What a Figure Cannot Tell You

The observation that a distribution is skewed to one side does not provide a mechanism for "why it is skewed." Nor does the observation that values in one condition are more widely spread tell you whether the source of variation is biological, part of the measurement process, or due to batch composition. Do not write an observation from a figure and an explanation of its cause in the same sentence.

Also, do not conclude that conditions are the same merely because two boxplots overlap. Overlap only shows a relationship between visual ranges; the effect size, uncertainty, and design judgments required by the research question call for a separate analysis. Conversely, boxes appearing separated does not establish a causal effect either.

When saving a figure, record the following metadata along with it.

This record exists so that visual results can be revisited later and the assumptions under which they were produced can be confirmed.

Common Failures and How to Check for Them

• Do not exaggerate differences by truncating axes or hiding units.
• Do not read the virtual positions from jitter as actual measured values.
• Do not automatically remove points outside the box plot.
• Do not display technical replicates and biological replicates as independent points in the same color.
• Do not hide the number of observations and missing values per group in the figure caption.

Key Points

• Dot plots show individual observations, histograms show frequencies per bin, and box plots show a summary of the distribution.
• An outlier is not a command to delete, but a signal to re-check the record, the measurement, and the transformation.
• Visualization requires axes, units, the unit of observation, and the meaning of missing values and replicates.
• Figures check data quality and questions, but they do not automatically produce causal conclusions.

To the Next Topic

The next installment covers missing values, duplicates, and transformations. It connects the blanks and abnormal ranges found in the plots to the assumptions and records used to handle them.

References

• pandas User Guide: https://pandas.pydata.org/docs/user_guide/index.html
• NumPy Reference: https://numpy.org/doc/stable/reference/

The scenarios and visualization code in this installment are educational material independently written by BioStatPy.