Measures of Effect
Ratios of risk/odds. Good for strength of association, but a relative figure hides the baseline risk - a 50% RRR means little if the event is very rare.
The actual difference in event rates. Reflects real-world magnitude of benefit and depends on baseline risk.
1/ARR - the most clinically intuitive measure: how many patients you treat to prevent one event. Lower NNT = bigger effect.
Overview & Introduction
A measure of effect puts a single number on the question "how much does this exposure or treatment change the outcome?" In critical appraisal - a core skill the FRCS/FRACS basic-science and viva stations test - your job is to pick the correct measure for the study design, interpret it with its confidence interval, and decide whether a statistically significant result is also clinically worthwhile.
The measures fall into four families, and the examiner expects you to reach for the right one:
- Relative (ratio) measures - relative risk, odds ratio, hazard ratio. They express the proportional change and convey strength of association, but they deliberately strip out the baseline risk.
- Absolute measures - absolute risk reduction (ARR) and the number needed to treat (NNT). They put the effect back in the context of baseline risk and reflect real-world benefit.
- Time-to-event measures - the hazard ratio, used when when an event happens matters (e.g. time to implant revision), not just whether it happened.
- Continuous-outcome measures - the mean difference or a standardised effect size (Cohen's d) for scores such as pain, range of motion or function.
The single most common viva error is quoting a relative figure ("a 50% reduction!") without the absolute one. The instant you hear or say a relative risk reduction, ask for the ARR and NNT - that is the move the examiner is waiting for.
Concepts: Relative Risk & Odds Ratio
Both express how much an exposure/treatment changes the chance of an outcome, but they are computed differently and arise from different designs:
- Relative risk (RR) = (risk in exposed) / (risk in unexposed), where risk = events/total in that group. RR is the natural measure of a cohort study or RCT, where you can measure true incidence.
- Odds ratio (OR) = (odds in exposed) / (odds in unexposed), where odds = events/non-events. OR is the measure of a case-control study (where incidence cannot be measured) and of logistic regression.
For both, 1 = no effect, less than 1 = protective, greater than 1 = increased risk. The OR approximates the RR when the outcome is rare; when the outcome is common, the OR is further from 1 than the RR and can exaggerate the apparent effect.
Examiners frequently ask the difference between odds and risk before letting you near an odds ratio. Risk (probability) is events divided by the total - if 20 of 100 patients have the event, risk = 0.20 (20%). Odds is events divided by non-events - the same data give odds = 20/80 = 0.25. So odds and risk diverge as the outcome becomes common, and odds are always larger than the corresponding risk; they converge only when the event is rare. The conversion is odds = risk / (1 - risk) and risk = odds / (1 + odds).
The practical trap follows: an odds ratio is not a "times more likely" statement about risk. An OR of 2 does not mean the outcome is twice as likely (that would be a risk ratio of 2) - it means the odds are doubled, which overstates the risk ratio when the outcome is common. State an OR as a change in odds, and if the examiner wants "how much more likely," either reach for the relative risk or note that OR approximates RR only when the event is rare.
Examiners love to hand you a 2x2 contingency table and ask for RR and OR. Lay it out the same way every time, then read the formulae straight off the cells.
- Definition
- a = events, b = non-events
- Formula
- risk = a/(a+b); odds = a/b
- Definition
- c = events, d = non-events
- Formula
- risk = c/(c+d); odds = c/d
- Definition
- risk ratio (cohort/RCT)
- Formula
- [a/(a+b)] / [c/(c+d)]
- Definition
- odds ratio (case-control/logistic)
- Formula
- (a/b)/(c/d) = ad/bc
- Definition
- outcome rare (a much smaller than b; c much smaller than d)
- Formula
- OR converges on RR
- Definition
- outcome common
- Formula
- OR sits further from 1 than the true RR
A meta-analysis of intravenous tranexamic acid in revision joint arthroplasty reported the odds of needing a blood transfusion as OR = 0.20 (95% CI 0.11-0.34) with TXA versus control - i.e. roughly an 80% reduction in the odds of transfusion, and the confidence interval lies entirely below 1, so the effect is statistically significant. By contrast the OR for venous thromboembolism was 0.57 (95% CI 0.13-2.42) - the CI crosses 1, so that difference is not significant. This shows how the point estimate and its CI together convey both direction/size and significance.
Absolute Measures, RRR & NNT
- Formula
- events / total (control)
- What it tells you
- Baseline risk
- Formula
- events / total (treated)
- What it tells you
- Risk on treatment
- Formula
- CER - EER
- What it tells you
- Real difference in risk
- Formula
- ARR / CER = 1 - RR
- What it tells you
- Proportional reduction (hides baseline)
- Formula
- 1 / ARR (round up)
- What it tells you
- Patients treated to prevent one event
- Formula
- 1 / absolute risk increase
- What it tells you
- Patients treated to cause one harm
A relative risk reduction can sound dramatic while the absolute benefit is tiny. If a treatment cuts risk from 2% to 1%, the RRR is 50% but the ARR is only 1%, giving an NNT of 100 (treat 100 patients to prevent one event). The same 50% RRR applied to a baseline of 40% to 20% gives an ARR of 20% and an NNT of 5. Always pair the relative figure with the absolute one to judge real clinical value - and weigh NNT against the NNH for adverse effects.
Hazard Ratio, Effect Size & Confidence Intervals
For survival / time-to-event outcomes (e.g. time to implant revision), the hazard ratio (HR) compares the instantaneous event rate between groups over follow-up (from Cox regression / Kaplan- Meier). Interpreted like other ratios: HR = 1 no difference, less than 1 lower hazard, greater than 1 higher hazard. It uses the timing of events, not just whether they occurred.
The confidence interval is the front end of a hypothesis-testing framework examiners expect you to define:
- P-value: the probability of obtaining a result at least as extreme as observed IF the null hypothesis were true. It is not the probability that the null is true, nor that the result is "due to chance" — a subtle but examined distinction. A CI excluding the null and p under 0.05 carry the same information.
- Type I error (α): a false positive — concluding there is an effect when there is none; the significance threshold (conventionally 0.05) is the accepted type I error rate. Testing many outcomes inflates it (multiplicity).
- Type II error (β): a false negative — missing a true effect; conventionally accepted at 0.2.
- Power = 1 − β (conventionally ≥80%): the probability of detecting a true effect of a given size; it rises with sample size and effect size and falls with variability, and drives the a-priori sample-size calculation.
- The practical corollary: a non-significant result in an underpowered study (wide CI) is "absence of evidence, not evidence of absence" — it does not prove equivalence.
Clinical Relevance: Reading Effect Measures in Practice
A statistically significant result (CI excludes the null, p less than 0.05) only means the effect is unlikely to be due to chance - it says nothing about whether the effect is big enough to matter. A large trial can make a trivial difference significant. The bridge is the minimal clinically important difference (MCID): the smallest change a patient actually notices. Always ask two questions in sequence - Is it significant? (look at the CI), then Is it clinically important? (compare the effect, and the lower end of its CI, against the MCID). A statistically significant mean difference in a pain score that is smaller than the MCID is real but clinically meaningless.
A reliable, examiner-pleasing routine when handed a result or abstract:
- Name the measure and check it fits the design - RR/ARR/NNT from a cohort or RCT; OR from a case-control study or logistic regression; HR from survival data; mean difference for continuous outcomes.
- Read the point estimate with its 95% CI - direction, magnitude, and whether the CI crosses the null (1 for ratios, 0 for differences). A narrow CI signals precision; a wide one signals an underpowered or small study.
- Demand the absolute numbers - convert a relative figure to ARR and NNT, because the same RRR means a very different real-world benefit at different baseline risks.
- Judge clinical importance - compare against the MCID and weigh the NNT against the NNH, cost, and patient burden before recommending the intervention.
The NNT is baseline-risk dependent, so it cannot be lifted from one population and applied to another. A treatment with an NNT of 5 in a high-risk group (e.g. elderly, high comorbidity) may have an NNT of 100 in a low-risk group for the same relative effect. In an exam this is the trap behind "the trial showed an NNT of X - would you apply it to this patient?" The correct answer is to re-derive the NNT for the patient's own baseline risk: NNT for the new population is approximately 1 / (baseline risk in that population multiplied by RRR).
On a forest plot, three things tell the whole story at a glance: which side of the line of no effect each square sits (direction), the width of each horizontal line (precision), and whether the pooled diamond touches the line (overall significance). Spread-out squares with little overlap also flag heterogeneity - a reason to be cautious about the pooled estimate.
When a meta-analysis combines effect measures, examiners expect the interpretive tools beyond "the diamond crosses the line":
- Heterogeneity is quantified by I² — the percentage of variation across studies due to real differences rather than chance (rough guide: under ~25% low, ~50% moderate, over ~75% high) — supported by Cochran's Q (χ²) test (which is itself low-powered). High I² undermines a single pooled estimate.
- Model choice: a fixed-effect model assumes one true common effect (appropriate only with low heterogeneity); a random-effects model assumes the true effect varies between studies, incorporates the between-study variance, gives wider confidence intervals, and is used when heterogeneity is present.
- The fragility index is an examined caution for small "significant" RCTs: the minimum number of patients whose outcome would have to change to make a significant result non-significant. Many orthopaedic trials have a low fragility index (a few events), so a result that is statistically significant can still be fragile — read it alongside the effect size and CI.
Mnemonics & Memory Aids
RATIO
Hook:RATIO: RR for risk, OR for odds, both null at 1, CI tells significance.
ARR-NNT
Hook:ARR drives NNT (1/ARR); a big RRR with a tiny ARR means a huge NNT.
Clinical Decision Scenarios
Practise clinical reasoning and management decisions out loud
“What is the difference between relative risk and odds ratio, and when is each used? When does the odds ratio mislead?”
“A trial reports a 50% relative risk reduction with a new treatment. Why might that be misleading, and how do absolute risk reduction and number needed to treat help?”
“A registry study reports a hazard ratio of 1.3 (95% CI 1.1-1.4) for revision with uncemented versus cemented total knee replacement. Interpret this, and explain how it differs from an odds ratio and what statistical significance here does and does not tell you.”
“What is the difference between odds and risk, and why does it matter when you interpret an odds ratio?”
Ratio measures
- RR = risk ratio (cohort/RCT); OR = odds ratio (case-control/logistic)
- 1 = no effect; below 1 protective; above 1 harmful
- OR approximates RR only when outcome rare; CI crossing 1 = non-significant
Absolute & NNT
- ARR = CER - EER; RRR = ARR/CER = 1 - RR
- NNT = 1/ARR (round up); NNH = 1/ARI
- Big RRR + small ARR = large NNT (baseline-dependent, not transferable)
Other effect measures
- Hazard ratio (HR) for time-to-event (survival); null at 1
- Mean difference / Cohen's d (0.2/0.5/0.8 = small/medium/large) for continuous
- Compare with MCID for clinical meaning
Confidence intervals
- Ratio measures null = 1; difference measures null = 0
- CI includes null means non-significant; narrow CI means precise
- CI more informative than a bare p-value; shown on a forest plot
Evidence & Key Studies
Intravenous tranexamic acid use in revision total joint arthroplasty: a meta-analysis
- Across 7 studies (930 patients), intravenous TXA significantly reduced the odds of transfusion in revision arthroplasty: OR 0.20 (95% CI 0.11-0.34) - the CI entirely below 1, so significant.
- VTE risk did not differ significantly: OR 0.57 (95% CI 0.13-2.42) - the CI crosses 1, so non-significant - a clean illustration of interpreting odds ratios and confidence intervals.
- Also reports mean differences (continuous effect sizes) for haemoglobin drop and RBC units transfused.
Platelet-rich plasma versus hyaluronic acid for knee osteoarthritis: a systematic review and meta-analysis of randomized controlled trials
- Across 18 level-1 studies (811 PRP vs 797 HA patients), mean WOMAC improvement was significantly higher with PRP (44.7%) than HA (12.6%) - a between-group continuous effect, not a ratio measure.
- Illustrates that for continuous outcomes the magnitude (mean difference / effect size) is the key effect measure, complementing significance testing.
- Demonstrates how effect measures are reported alongside p-values in orthopaedic level-1 evidence.
Association between fixation type and revision risk in total knee arthroplasty patients aged 65 years and older: a cohort study of 265,877 patients from the Nordic Arthroplasty Register Association 2000-2016
- In 265,877 TKAs, uncemented fixation carried a higher revision risk than cemented: adjusted hazard ratio 1.3 (95% CI 1.1-1.4) - the CI lies entirely above 1, so significant.
- Derived from Cox regression on time-to-revision data - the worked hazard-ratio example complementing the odds-ratio and mean-difference cards.
- 10-year Kaplan-Meier survivorship was 96% (cemented) versus 94% (uncemented), showing how a survival curve and a hazard ratio describe the same time-to-event data.
These three cards are chosen to illustrate the three commonest effect-measure types in orthopaedic evidence: an odds ratio with its confidence interval (Kuo et al., DOI), a continuous mean difference / effect size (Belk et al., DOI), and a hazard ratio from time-to-event registry data (Irmola et al., DOI). The underlying formulae and definitions (RR, OR, ARR, RRR, NNT, HR, Cohen's d) are standard biostatistics presented as mathematical relationships, not findings of any single study. (See also our Diagnostic Test Statistics and Study Design topics.)