Epidemiology — Data Analysis, Treatment Outcomes and Study Evaluation
A new cancer drug reduces mortality by 50% — sounds dramatic. But if the baseline risk was 2%, the absolute reduction is only 1 percentage point. Understanding the difference between relative and absolute risk is what separates critical reading of medical evidence from being misled by statistics.
Download this lesson's worksheet
Use the PDF for classwork, homework or revision. It includes key ideas, activities, questions, an extend task and success-criteria proof.
Kidney Disease
"This Drug Cuts Heart Attack Risk by 40%" — Should You Take It?
A newspaper headline reads: "New cholesterol drug cuts heart attack risk by 40%." The drug costs $200/month and has moderate side effects in 10% of users.
The actual trial data: in the placebo group, 5 out of every 1,000 patients had a heart attack over 5 years. In the drug group, 3 out of every 1,000 patients had a heart attack over 5 years.
Before reading on:
Q1: The 40% figure is the relative risk reduction. Calculate the absolute risk reduction (the actual difference in risk between the two groups). How does the absolute figure compare to the relative figure in terms of what it means for an individual patient?
Q2: If 1,000 patients take this drug for 5 years, how many heart attacks are prevented? How does this change your assessment of the drug's value?
Know
- How to calculate relative risk, absolute risk reduction, and NNT from trial data
- How to read and interpret a basic survival curve
- The hierarchy of evidence from case reports to systematic reviews
- What a p-value represents and its limitation as the only measure of significance
Understand
- Why relative risk reduction can be misleading without absolute risk context
- Why NNT is more useful for clinical decision-making than relative risk
- Why systematic reviews and meta-analyses provide stronger evidence than individual studies
- How to identify limitations of studies and what they prevent you from concluding
Can Do
- Calculate RR, ARR, RRR, and NNT from a 2×2 table or trial data
- Interpret a survival curve and identify what the gap between curves represents
- Evaluate whether a study's conclusions are supported by its data and design
- Identify when a statistically significant result may not be clinically meaningful
Connect this concept back to the broader homeostasis and disease framework you have built across the course.
Relative Risk, Absolute Risk Reduction and Number Needed to Treat
Try this: Enter values into the 2×2 table or load an example, then observe how RR, attributable risk, and odds ratio change. Try RR = 1, RR > 1, and RR < 1 to see how the interpretation shifts.
This calculator demonstrates why relative risk alone can be misleading without absolute risk context.
Relative Risk tells you how many times more likely the exposed group is to develop the outcome. RR = 1 means no association. RR > 1 indicates increased risk. RR < 1 indicates protection. Always interpret RR alongside absolute risk reduction and NNT to understand real-world clinical significance.
Relative risk tells you how much more or less likely an outcome is in one group compared to another — expressed as a ratio. Absolute risk reduction tells you the actual size of that difference in real-world terms. Number needed to treat translates that difference into a clinically meaningful statement about how many patients benefit. All three are needed to evaluate a treatment honestly.
Epidemiology showing study types, measures and evaluation
Bradford Hill criteria for establishing causation
Worked Example — Statin Drug for Heart Disease Prevention
An RCT follows 10,000 patients (5,000 statin, 5,000 placebo) for 5 years. Results:
Statin group: 100 heart attacks out of 5,000 = risk of 0.02 (2%)
Placebo group: 150 heart attacks out of 5,000 = risk of 0.03 (3%)
RR = 0.02 ÷ 0.03 = 0.67 — the statin group has 67% of the risk of the placebo group (a 33% lower relative risk).
ARR = 0.03 − 0.02 = 0.01 (1%) — the statin reduces absolute heart attack risk by 1 percentage point over 5 years.
RRR = 0.01 ÷ 0.03 = 33% — the statin reduces relative risk by 33%.
NNT = 1 ÷ 0.01 = 100 — 100 patients must take the statin for 5 years to prevent one additional heart attack.
Interpretation: A headline saying "statins reduce heart attack risk by 33%" is technically accurate (RRR) but can be misleading — the absolute reduction is only 1%. Whether NNT = 100 is acceptable depends on the drug's cost, side effects, and the severity of the outcome prevented. For a condition as serious as heart attack, NNT = 100 may well be worthwhile. For a minor condition, it may not be.
Why relative risk can mislead
Relative risk amplifies small effects in low-risk populations. "This supplement reduces cancer risk by 50%" sounds impressive — but if the baseline risk is 0.002% (2 in 100,000), a 50% reduction means an absolute reduction from 0.002% to 0.001%. NNT would be 100,000 — you would need to treat 100,000 people to prevent one cancer. The relative figure is truthful but decontextualised from real-world importance.
Conversely, an ARR of 5% (NNT = 20) represents a very effective treatment — for every 20 people treated, one extra bad outcome is prevented. In clinical medicine, NNT values below 10 are considered highly effective; 10–100 moderately effective; above 100 marginal.
Survival Curves — Reading Kaplan-Meier Graphs
A survival curve (Kaplan-Meier plot) shows the proportion of a study population that has not yet experienced the primary outcome (often death, but also disease recurrence, hospitalisation, or other events) over time. They appear in almost every major clinical trial and in many HSC exam questions about epidemiological data.
How to read a survival curve
- Y-axis: Proportion of participants who have not yet experienced the outcome (usually 0–1 or 0–100%). Starts at 1.0 (100% event-free) and falls over time as participants experience the outcome.
- X-axis: Time (months, years).
- Multiple lines: Each line represents a different group (e.g. treatment vs placebo; smokers vs non-smokers). A line that falls more steeply = more events happening faster = worse outcome.
- Gap between lines: The vertical distance between lines at any time point represents the difference in survival probability between groups at that time. A widening gap over time suggests the treatment benefit increases; a converging gap suggests diminishing benefit.
- Plateau: A line that flattens indicates no further events are occurring — either participants have reached long-term survival, the study has ended, or participants are being lost to follow-up.
- Censoring marks (tick marks on lines): Small vertical marks on the survival line indicate participants who were 'censored' — lost to follow-up, withdrew, or the study ended before they experienced the outcome. These people's outcomes are unknown.
Worked Example — Interpreting a Survival Curve
A melanoma trial shows two survival curves over 5 years. The immunotherapy group starts at 1.0 and falls gradually to 0.52 (52% surviving at 5 years). The chemotherapy group starts at 1.0 and falls more steeply to 0.28 (28% surviving at 5 years). The curves diverge from 6 months onward.
What can you conclude:
At 5 years, 52% of immunotherapy patients were still alive vs 28% of chemotherapy patients — a difference of 24 percentage points (absolute difference in 5-year survival).
The curves diverge from 6 months — suggesting immunotherapy benefit begins early and increases over time. This divergence pattern is consistent with immunotherapy's mechanism (stimulating durable immune responses that continue killing cancer cells).
You cannot conclude that all immunotherapy patients will survive long-term — the curves show 48% of immunotherapy patients also died within 5 years. What the study shows is that immunotherapy more than doubled the proportion surviving at 5 years compared to chemotherapy.
The Evidence Hierarchy — From Single Case to Systematic Review
Try this: Select each study description card, then place it in the correct study type bin. Check your answers when all six cards are placed.
Recognising study designs from their description is an essential HSC skill tested directly in exam questions.
Cohort studies follow exposed groups forward in time. Case-control studies compare past exposures between cases and controls. Cross-sectional studies measure exposure and disease at one time point. Recognising these designs from their description is essential for evaluating epidemiological evidence.
In medicine and public health, evidence is graded by quality. Evidence from a single patient case report is informative but cannot establish general truths. Evidence from a well-conducted systematic review of dozens of RCTs provides the most reliable basis for clinical decisions. Understanding this hierarchy allows you to evaluate claims critically — and to recognise when media reports cherry-pick weak evidence to make strong claims.
| Level | Study type | Strength | Limitation | Example |
|---|---|---|---|---|
| 1 (strongest) | Systematic review and meta-analysis of RCTs | Pools results of multiple high-quality trials; greatest statistical power; controls for individual study quirks | Quality depends on quality of included studies; publication bias can distort results | Cochrane review of statin trials |
| 2 | Single well-designed RCT | Randomisation controls confounders; establishes causation | May not generalise to all populations; can be underpowered | UKPDS trial — metformin for Type 2 diabetes |
| 3 | Cohort study | Prospective; establishes temporal sequence; large populations possible | Observational — cannot control all confounders | Nurses' Health Study — diet and cancer |
| 4 | Case-control study | Efficient for rare diseases; retrospective | Recall bias; cannot establish incidence | Case-control study of HPV and cervical cancer |
| 5 | Cross-sectional study | Cheap; generates hypotheses | Cannot establish temporal sequence | National Health Survey — diet and diabetes prevalence |
| 6 (weakest) | Case report / expert opinion | Identifies novel phenomena; hypothesis-generating | No comparison group; no statistical analysis; highly susceptible to bias | "Patient who ate X recovered from Y" |
Statistical significance vs clinical significance
A p-value below 0.05 (the conventional threshold for 'statistical significance') means there is less than a 5% probability of observing the result by chance if the null hypothesis (no effect) were true. It does NOT mean the effect is clinically important. With very large sample sizes, even tiny trivial differences become statistically significant.
Example: A study of 500,000 patients finds that a new drug reduces blood pressure by an average of 0.3 mmHg compared to placebo (p = 0.001 — highly statistically significant). A 0.3 mmHg reduction in blood pressure is clinically meaningless — no patient would benefit detectably from such a small change. The study found a real effect, but not a useful one. Statistical significance tells you whether an effect exists; clinical significance (effect size, ARR, NNT) tells you whether it matters.
Evaluating a Study — A Systematic Approach
The HSC regularly asks students to evaluate study quality. This is not about finding flaws for the sake of it — it is about identifying what a study can and cannot establish, so that claims based on its results can be appropriately qualified.
Key evaluation criteria
- Study design: Is it the right design for the question? (RCT for intervention; cohort for long-term exposure; case-control for rare disease).
- Sample size: Is it large enough to detect a real effect? Small samples are underpowered — they may miss real effects (false negative) or produce spurious results.
- Representativeness: Does the study population reflect the target population? RCTs often exclude elderly, pregnant, or multi-morbid patients — limiting generalisability.
- Blinding: Were participants and/or researchers blind to treatment allocation? Single-blind (participants unaware); double-blind (participants and assessors unaware). Unblinded studies are more susceptible to placebo effect and assessment bias.
- Control group: Is there an appropriate comparison group? Placebo vs active control vs no treatment — the choice affects what conclusions can be drawn.
- Follow-up: Was the follow-up period long enough? Diseases with long latency periods (cancer, cardiovascular disease) require years of follow-up — short studies miss delayed outcomes.
- Confounding: Were potential confounders identified and controlled? In observational studies, residual confounding is always a risk.
- Outcome measurement: Were outcomes measured objectively and consistently? Subjective outcomes (pain, quality of life) are more susceptible to bias than objective outcomes (death, laboratory values).
- Statistical analysis: Was the appropriate statistical method used? Were confidence intervals reported alongside p-values?
Worked Example — Evaluating a Hypothetical Study
Study: A 6-week RCT of 200 patients found that a new anti-inflammatory drug reduced self-reported knee pain by 35% more than placebo (p = 0.03). The study was single-blind (patients did not know which group they were in, but researchers did). Patients with severe kidney disease were excluded.
Strengths: RCT design — randomisation controls for most confounders. Appropriate study design for testing a new treatment.
Limitations to note: (1) Single-blind — researchers who knew treatment allocation could unconsciously bias their assessments of patient-reported pain (assessment bias). Double-blinding would be stronger. (2) 6 weeks is short — many musculoskeletal conditions improve spontaneously over 6 weeks (regression to the mean). A longer trial would be more convincing. (3) Self-reported pain is subjective — placebo effect is substantial for pain outcomes even with blinding. (4) Excluded severe kidney disease patients — results may not generalise to this group who may have different drug metabolism. (5) p = 0.03 is statistically significant but close to the threshold — with a small sample (200), there is more risk this reflects sampling variation.
How NNT Changed the Way Doctors Prescribe Statins
The Heart Protection Study (HPS), published in 2002, was one of the largest cardiovascular trials ever conducted — 20,536 patients with existing cardiovascular disease or high risk, followed for 5 years. It found that simvastatin reduced major vascular events (heart attacks, strokes, revascularisation procedures) by about 24% relative risk reduction compared to placebo.
The headline figure — 24% relative risk reduction — was used extensively to promote statin prescribing. But the absolute figures were equally important: the event rate fell from approximately 25.2% in the placebo group to 19.8% in the statin group — an ARR of 5.4 percentage points, giving an NNT of approximately 19 over 5 years. This means treating 19 high-risk patients with simvastatin for 5 years prevents one additional major vascular event.
For high-risk patients with existing cardiovascular disease, NNT = 19 is considered highly clinically significant — statins rapidly became standard of care for this group. But when the same relative risk reduction (24%) was applied to lower-risk primary prevention populations (people without existing CVD), the absolute event rate in the placebo group was much lower (~5% over 5 years), producing an ARR of only ~1.2% and an NNT of ~83. The same drug, the same relative risk reduction, but very different absolute benefit — which is why prescribing decisions for primary prevention are more nuanced than for secondary prevention. This is precisely why NNT matters.
Priority Misconceptions — Data Analysis and Study Evaluation
"A large relative risk reduction means the treatment is very effective." — Relative risk reduction must always be considered alongside the absolute baseline risk. A 50% relative risk reduction from 0.002% to 0.001% (ARR = 0.001%, NNT = 100,000) is far less clinically meaningful than a 25% relative risk reduction from 20% to 15% (ARR = 5%, NNT = 20).
"A p-value below 0.05 means the result is important." — Statistical significance (p < 0.05) means the result is unlikely to be due to chance. It says nothing about clinical importance. With large enough samples, trivially small and meaningless differences become statistically significant. Always assess clinical significance (effect size, ARR, NNT) alongside statistical significance.
"A survival curve that falls to zero means all patients died." — A survival curve that reaches zero means all patients in the study eventually experienced the primary outcome within the follow-up period. However, studies usually end before all participants experience the outcome — a curve that plateaus does not mean those patients are cured; it means the follow-up period ended. Censoring marks indicate patients who left the study before the outcome — their fate is unknown, not assumed to be survival.
"A systematic review is just a literature review." — A systematic review uses pre-specified, reproducible methods to identify and critically appraise ALL relevant studies on a question, minimising selection bias in which studies are included. A narrative (regular) literature review is selective — the author chooses which studies to discuss, which can introduce bias. Meta-analysis within a systematic review statistically pools study results. These methodological distinctions place systematic reviews at the top of the evidence hierarchy.
"If a study found no effect (p > 0.05), the treatment definitely doesn't work." — A non-significant result means there was insufficient evidence to reject the null hypothesis — not that the null hypothesis is true. A small study may be underpowered to detect a real but modest effect. The absence of evidence is not evidence of absence. Confidence intervals around the null result tell you more than the p-value alone — a wide confidence interval crossing zero indicates uncertainty, not definitive null effect.
Image Slot 1: Side-by-side comparison of relative risk reduction vs absolute risk reduction using two scenarios: (A) High-risk population — 20% vs 15% event rate — ARR = 5%, NNT = 20; (B) Low-risk population — 0.4% vs 0.3% event rate — ARR = 0.1%, NNT = 1000. Both have the same RRR = 25%. Visual should show the same headline ("25% risk reduction") but starkly different real-world impact.
Image Slot 2: Annotated Kaplan-Meier survival curve with labels for: y-axis (proportion event-free), x-axis (time in months/years), two diverging lines (treatment vs control), the gap between lines at specific time points (annotated with values), censoring tick marks, and a shaded region showing the difference in 5-year survival. Arrows pointing to key features with explanatory notes.