How to Read Medical Big Data

Why context matters more than volume

Medical data are not simply tables with many rows. Clinical records, diagnosis and medication codes, laboratory results, images, notes, and biosignals arise from different processes. The same number can mean something different depending on the hospital, instrument, unit, patient, and point in the care pathway at which it was measured.

A sound medical-data study therefore asks what is being predicted or explained, how each value was generated, and whether that information was actually available at the intended time of analysis before asking how much data were collected. A large dataset may create statistical opportunities, but it does not automatically remove miscoding, selection bias, temporal contamination, or differences between institutions.

The layers of medical data

These distinctions are not merely about file formats. Whether a test was ordered reflects not only the patient's state but also a clinician's decision and the institution's workflow. Missingness is therefore not always random; the absence of a test may itself carry a trace of clinical decision-making.

Preprocessing is not neutral cleaning

Rules for grouping or excluding diagnosis codes, handling repeated visits, harmonizing laboratory units, removing outliers, and addressing missing values can change the meaning of both the cohort and its features. Across institutions or time periods, instruments, reagents, reference ranges, coding and billing practices, clinical protocols, patient populations, and labeling procedures may all change.

Researchers should preserve more than the final analysis table. They should record the source system, extraction date, cohort definition, unit conversions, exclusion rules, and manual adjudication procedures. Data provenance is not an appendix; it is essential for interpreting the result again.

A small example — the wrong path to high performance

Suppose a model is intended to predict a complication using information available at admission. If prescriptions or test results recorded after the outcome occurred enter the predictors, the model is not forecasting the future; it is reading traces of an outcome that has already become visible. This is one form of temporal or label leakage.

Randomly splitting rows can also place different visits from the same patient in both training and test sets. If the goal is to generalize to new patients, the patient should be the splitting unit so that this overlap is prevented. If the question concerns performance over time, development on an earlier period and evaluation on a later period may better match the intended use.

Define the research question and prediction time
                    ↓
Fix the cohort, index time, and observation window
                    ↓
Split with patient, site, and time dependencies in mind
                    ↓
Preprocess and develop the model
                    ↓
Internal validation + external validation at another site or time

There is no single correct split for every study. The method should match the target population and use scenario and should be reported explicitly.

External validation is more than reproducing one number

A high AUC at one hospital does not automatically have the same meaning at another. Patient mix, laboratory instruments, measured variables, care pathways, and label definitions may differ. In a published DIC prediction study, the development and external-validation hospitals used different laboratory instruments and profiles, and the investigators documented those differences when evaluating the model.

External validation should examine not only performance metrics but also eligible patients, exclusion criteria, timing of predictor measurement, handling of missing data, calibration, and differences between institutions. Even strong predictive performance does not establish patient benefit or safe integration into clinical workflow; those require separate prospective evaluation.

Privacy and governance are part of study design

De-identification is an important safeguard, but it does not remove every risk. A medical-data study also needs defined access rights, permitted purposes, retention periods, auditability, security, consent or another lawful basis, representation of affected groups, and lines of accountability.

WHO guidance recommends addressing autonomy, safety and public interest, transparency, accountability, inclusiveness and equity, and sustainability throughout the design, development, and use of AI for health. This is not legal advice for a particular jurisdiction. Each study must separately check applicable law, institutional policy, and IRB or equivalent review requirements.

Common misconceptions

Does more data reduce bias?

Not necessarily. Collecting more records produced by the same workflow and selection process can also amplify an existing bias.

Should every missing value simply be filled in?

No. First understand why the value is absent and whether it would have been available at prediction time, then report the handling method and reasons for omission.

Can de-identified data be used freely?

No. Re-identification risk, access controls, approved purpose, contracts, and applicable law still matter.

Does a high AUC establish clinical usefulness?

No. Calibration, external generalizability, comparators, patient outcomes, safety, and effects on real workflow require separate evaluation.

Current evidence and limits

• This is a Concept about designing and interpreting medical-data research.
• It does not provide a diagnosis, individual treatment advice, or institution-specific legal or security procedures.
• Leakage controls and splitting strategies depend on the research question and data structure and should be specified in the protocol and analysis plan.

Connected Concepts / Story

• Concepts: cohort, missing data, data leakage, external validation, clinical prediction model
• Story candidate: tracing the data-generation process when the same model performs differently at two hospitals

References

• Yoon et al. Machine learning-based diagnosis for disseminated intravascular coagulation, PLOS ONE (2018)
• TRIPOD+AI statement, BMJ (2024)
• A framework for understanding label leakage in machine learning for health care, JAMIA Open (2023)
• WHO Ethics and governance of artificial intelligence for health (2021)