Count Matrices, Normalization, and Batch Effects

Upon completing this topic

You will be able to verify the meaning of rows, columns, and counts in a count matrix, and distinguish between raw counts and normalized values. You will also learn to check whether batches are confounded with biological conditions, and to avoid the interpretation that normalization resolves all batch-related issues.

What is a single cell in the matrix?

In omics tables, rows often represent features and columns represent samples, but one should not assume this orientation based solely on file names. Verify whether a cell value represents raw read counts, already transformed abundance, or the result of some filtering. The relationship between sample IDs and biological replicates and technical replicates should also be recorded in the schema.

Raw counts and normalized values

Raw counts are values directly aggregated during the data generation and quantification process, while normalized values may be the result of transformations reflecting library size or other reference criteria. Do not mix the two values in the same column; leave provenance records of the transformation formula, tool, version, and input.

Normalized values may be useful for visualization or specific comparisons, but they cannot be automatically used as input for all models. For count-based models, verify the input scale expected by the model.

Viewing batches alongside the design

A batch is a set of units that share a common process, such as a measurement date, instrument, plate, or processing group. If batches and biological conditions completely overlap, it may be statistically difficult to separate the two effects. Applying correction code post hoc does not automatically eliminate confounding in the design.

sample_id | biological_condition | batch_id | replicate_id | value_type
S01       | A                    | B01      | R01          | raw_count

The first thing to check in this table is not the model function, but whether each condition was observed across multiple batches.

Returning to the research question for interpretation

Normalization is a computational step to convert values into a comparable representation. This does not mean it separates measurement bias, batch confounding, replicate errors, and biological differences. The analysis documentation should record raw inputs, normalization, batch processing, and the final model separately.

Common failures

• Do not refer to normalized values as raw counts.
• Do not automatically use other scales like TPM as input for count models.
• Do not write that a design where batches and conditions overlap can be resolved with a single line of code.
• Do not confuse the number of samples with the number of replicates.

Key takeaways

• First, verify the meaning of the matrix rows, columns, and individual cells.
• Raw counts and normalized values have different input scales.
• Check batches alongside the design and provenance.
• Normalization is not a procedure that eliminates all biases.

Next topic

After resolving the survival data blocker, connect to stat-022 and 023, and the next writable topic will be PCA and high-dimensional QC.

References

• NCBI GEO: https://www.ncbi.nlm.nih.gov/geo/info/
• ENCODE citing guidance: https://www.encodeproject.org/help/citing-encode/
• Bioconductor OSTA experimental design: https://bioconductor.org/books/release/OSTA/pages/bkg-exp-design.html

The matrix schema and explanations in this chapter were written independently by BioStatPy.