Randomization, Blocking, and Matching

Upon completing this topic

You will be able to distinguish between condition assignment and analysis, and explain how randomization, blocking, and matching address different sources of variation. You will also be able to create a checklist that aligns the assignment unit and the analysis unit as defined in the study design.

Questions are designed before analysis

When comparing the outcomes of Condition A and Condition B, the analyst receives already observed data. However, the fairness of the comparison depends on how the conditions were assigned. Randomization is a design principle for managing chance differences during the assignment process; it is not a property generated post hoc by p-value calculation functions.

Three design tools

Randomization selects one possible assignment according to a predefined rule. Blocking groups units based on known sources of variation (such as batch or time) before comparison, and then assigns conditions within each block. Matching pairs units that are similar or linked, enabling within-pair comparisons.

These three tools do not share the same purpose. Blocks are not nuisance factors that must be removed during analysis, and pairs cannot be formed solely because IDs are similar. The design record must document the level at which conditions were assigned.

Design schema

unit_id: 독립 분석 단위
block_id: 사전에 정한 공통 변동 요인
pair_id: 설계상 연결된 두 단위
assigned_condition: 무작위화 결과
outcome: 사후 측정값

The mere presence of block_id and pair_id does not automatically imply their inclusion in the analysis. Documentation must specify which source of variation was reduced, whether the units are complete, and how they are reflected in the analysis.

Questions for selecting the three tools

Randomization is used to reduce the predictability of condition assignment and to create comparable assignment procedures. Blocking is designed to ensure comparisons occur within the same block when known sources of variation exist prior to condition assignment. Matching is used when two units have an explicit link, such as pre-post measurements on the same subject or similar baseline characteristics.

While blocking and matching both place similar units close together, they may differ in the level at which they are used in analysis and the unit of comparison. The manifest must document the criteria used and confirm that these were decided before viewing the results.

Interpreting through the research question

A good design creates conditions under which outcome differences can be interpreted, but it does not guarantee the elimination of all biases. Issues such as measurement error, attrition, and extrapolation may remain. The use of design tools and the magnitude of actual effects should be reported separately.

Common failures

• Do not use a fixed random seed as independent evidence of randomization.
• Do not select favorable blocks after viewing the results.
• Do not arbitrarily match values with the same ID.
• Do not leave the assignment unit and analysis unit different without documenting the reason.

Key takeaways

• Randomization addresses the assignment process, blocking addresses known sources of variation, and matching addresses comparisons of linked units.
• Design tools are not created post hoc by analysis functions.
• The assignment unit and analysis unit must be documented first.

Next topic

The next module covers factorial designs with two factors and interactions.

References

• Bioconductor OSTA experimental design: https://bioconductor.org/books/release/OSTA/pages/bkg-exp-design.html
• NIST/SEMATECH e-Handbook: https://www.itl.nist.gov/div898/handbook/

The schema and examples in this module were written independently by BioStatPy.