Background

The gold standard for measuring vaccine efficacy remains randomized controlled trials (RCTs). Their rigorous design, including double-blinding and placebo control, minimizes confounding variables and provides a solid basis for regulatory approval.

However, relying solely on RCTs to determine vaccine policy has significant limitations. The COVID-19 pandemic starkly illustrated this. With viral variants changing the antigenic landscape every few months, large-scale RCTs, which take years to complete, may be outdated by the time their results are available. Once a vaccine is approved, it becomes ethically challenging to withhold vaccination from the placebo group, thereby limiting opportunities to evaluate booster doses or updated vaccines through RCTs. The same dilemma has repeatedly arisen with annually updated vaccines, such as the influenza vaccine.

Amid these structural limitations, observational data from real-world settings has emerged as a valuable complement. In a recent Perspective in the New England Journal of Medicine, Arnold S. Monto of the University of Michigan and Helen Y. Chu of the University of Washington argue that clinical trials and observational studies should be integrated as a "strategy to combine the strengths of both approaches."

Key Findings

Maturation of Observational Study Methodologies

During the pandemic, the test-negative design (TND) became a key tool for evaluating vaccine effectiveness. The TND compares vaccination rates among vaccinated and unvaccinated individuals within a cohort of patients presenting to healthcare facilities with respiratory symptoms, thereby mitigating selection bias due to healthcare-seeking behavior. Most of the real-time evidence on the effectiveness of COVID-19 mRNA vaccines against variants, the waning of immunity over time, and the effects of booster doses has come from this design.

Cohort studies and linkage to administrative data have also become more sophisticated. By linking electronic health records (EHRs) and insurance claims data, observational cohorts of millions of individuals can be constructed, and propensity score matching or instrumental variable methods are used to adjust for confounding.

Complementarity of RCTs and Observational Studies

Monto and Chu emphasize that the two approaches should operate sequentially and complementarily, rather than competitively. RCTs establish the causal evidence for immunogenicity and safety in the initial approval phase, while observational studies track effectiveness, duration of immunity, and rare adverse events in the real-world population after approval. The RSV vaccine is a prime example. It was approved based on RCTs in older adults, and a dual structure is in place to monitor the effectiveness of the vaccine in the first season after administration and in subsequent seasons using observational data.

The influenza vaccine represents the area where this integrated strategy has been most extensively applied. Because the composition is changed annually, it is difficult to repeat RCTs, and multi-institutional TND surveillance systems, such as the CDC's US Flu VE Network, play a role in generating real-time estimates of effectiveness for each season.

Significance and Prospects

This Perspective highlights that the vaccine evaluation paradigm is shifting from reliance on a single methodology to an integrated, multi-layered evidence approach. The FDA has already implemented immunogenicity-based approval for updated COVID-19 vaccines and formalized the process of confirming clinical effectiveness through post-approval observational studies. This framework is likely to be extended to influenza, RSV, and other vaccines for pandemic preparedness.

However, the inherent limitations of observational studies remain. Unmeasured confounding variables, incomplete vaccination records, and biases due to differences in access to testing cannot be completely eliminated by design alone. The authors argue that standardized analysis protocols and the reproducibility of results in multi-institutional networks are key to improving reliability. Ultimately, a design that intentionally combines the internal validity of RCTs with the external validity of observational studies will be the key to ensuring both the speed and accuracy of vaccine policy in the face of rapidly changing pathogens.